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(54) Parallel distributed processing and operation processors 



(57) In a -system in which a plurality of network 
nodes (3. 4) are connected through a network (2), when 
a computation object (300-302) of a first network node 
(e.g. 3) calls a function (facility) of a computation object 
(400-402) of a second network node (4), the first net- 
work node obtains facility identification codes (numeral) 
of the object and the function from a remote reference 



table (310) and outputs the facility identification codes 
through the network (2) as a message together with the 
arguments of the function to the second network node 
(4). The second network node (4) uses the input facility 
identification code as a key to obtain the execution ad- 
dress of the facility from a local reference table (406) 
and executes that facility. 
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Description 

This invention relates to parallel distributed 
processing systems and methods that may, for example, 
be used for performing network-wide multimedia paral- 
lel processing. The invention also relates to operation 
processors. 

In recent years, there has been active development 
of ISMs (Information Super-Markets), that is. informa- 
tion supplying and utilizing service applications for ena- 
bling the utilization and circulation of multimedia infor- 
mation. In anTSM, to enhance the dialog among facility 
modules and the flexibility of the inter-module structure, 
an application execution environment enabling efficient 
network-wide multimedia parallel processing is re- 
quired. In response to this, a "parallel distributed proc- 
ess system" has been developed. 

In such a previously proposed parallel distributed 
processing system, a facility call is made among mod- 
ules by using the module name and facility name (meth- 
od name) indicated by a character string. Here, a 'facility 
call" means the call and use of a facility provided in an- 
other module in processing carried out in a certain mod- 
ule. 

A typical example of a parallel distributed process- 
ing system is the CORBA (Common Object Request 
Broker Architecture) - ORB (Object Request Broker). 
Here, the CORBA is a mechanism for intercommunica- 
tion of software operating at a plurality of computers 
connected to a network and serves as the basis for flex- 
ibly and efficiently constructing a distributed system. 
Further, ORB is software performing the communication 
among objects andJacilities to transfer a message re- 
ceived from a client object to a suitable server object via 
the network. 

In the CORBA-ORB. the management space of the 
objects is defined as the entire environment supported 
by the system. 

Further, in the CORBA-ORB. when a certain mod- 
ule calls a facility of another module, the facility name 
and arguments defined as symbols are transferred from 
the module originating the call to the module receiving 
the call. The module receiving the call is provided with 
a table of correspondence between the facility names 
and the execution addresses of the programs for exe- 
cuting the facilities. The module receiving the call ob- 
tains the execution address by searching through the 
correspondence table and executes the facility which is 
called. 

Further, a parallel distributed processing system 
wherein description is made of a'method of distributed 
allocation and execution of programs with which the us- 
er can describe the program without being conscious of 
which network node the module is arranged in is dis- 
closed in Japanese Unexamined Patent Publication 
(Kokai) No. 3-226856. 

In this method of distributed allocation and execu- 
tion of programs, a private directory for designating the 
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name of the network node in which each module is ar- 
ranged is provided. The description of an external call 
existing in a source program is automatically rewritten 
(amended) by the system while referring to this director 
5 ry. 

Further, the above-mentioned type of parallel dis- 
tributed processing system is often constructed using 
the Internet. In the Internet, an IP address and a DNS 
(domain name system) name are given to the nodes on 
io the network. The IP address is an identification number 
of each data processing device connected to the net- 
work and is expressed by a 32 bit number. Further, the 
DNS name is a name enabling differentiation of nodes 
on the network by a symbolic name having meaning to 
is the users. The network is divided into management 
ranges and a name given to each range. The DNS (do- 
main name system), which is a hierarchical naming 
mechanism using DNS names comprised by the series 
of names divided by periods establishes a hierarchy of 
20 the machine names in the Internet based on the TCP/l P. 
Note that, these IP addresses and DNS names 
must not overlap on the network, that is. through the 
world, and are centrally managed by a network informa- 
tion center (NIC) . 
25 in the conventional CORBA-ORB. however, since 
the module receiving the call uses the facility name in- 
dicated by the character string as a key to reler to the 
table and specify the execution address, and there are 
the problems that the cost for specifying the execution 
30 unit is large and the efficiency of the processing accom- 
panying the facility call is poor. 

Also, in the CORBA-ORB, the format of the argu- 
ments accompanying a facility call must be separately 
defined at the time of compilation. At this time, the data 
35 structure of the arguments which can be defined is lim- 
ited to simple one such as structure and array, thus there 
is the problem that complex data structures such as 
linked data structures cannot be used. 

Further, in the CORBA-ORB ; the module receiving 
JO the call must exist (be registered) in advance. Further, 
there is the problem that it is basically necessary to de- 
fine the format of the call facility and arguments at the 
time of compilation, thus when trying to dynamically 
download (additionally register) and use a module re- 
^5 ceiving a call, the work becomes troublesome. 

Further, in the CORBA-ORB, when the system ol 
synchronization for performing the facility call is differ- 
ent, a different programming interface (API) must be 
used, thus there is the problem that the description of 
so the program becomes complex. 

Further, in the conventional CORBA-ORB. the man- 
agement space o( the objects is defined as the entire 
environment supported by the system, therefore the us- 
er cannot freely define a management space having as 
55 a range of management only the required objects ac- 
cording to the particular purpose. 

Accordingly, there is a problem that the user must 
maintain uniqueness of the names even for unneces- 
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sary objects in the wide area of the system even when 
using only part of the objects for a particular purpose 
and otherwise manage the system, thus the load ac- 
companying this management is heavy. 

Further, in the CORBA-ORB, since it is necessary 
to maintain the uniqueness of names in the wide area 
of the system, there is the problem that the same names 
cannot be used among a plurality of applications sup- 
ported by the system, thus the degree of freedom of the 
names useable in each application is low. 

Further, since the objects are managed in the wide 
area of the system in this way. there is the problem that 
the number of the objects to be managed becomes 
enormous and the search time when accessing an ob- 
ject becomes long. 

Further, there also exists the problem that dynamic 
(incremental) space management accompanying addi- 
tion, deletion, etc. of the computation modules and so 
on is difficult. 

Further, in the method of distributed allocation and 
execution of programs disclosed in Japanese Unexam- 
ined Patent Publication (Kokai) No. 3-226856, since the 
description of the external call existing in the source pro- 
gram is amended while referring to the directory, there 
is a problem that, if the contents of the directory are 
changed in accordance with the change of the module 
allocation, it becomes necessary to newly compile the 
source program, so the processing accompanying the 
change is troublesome and cannot be quickly handled. 

Further, in the DNS, which is widely spreading as 
the name solution mechanim for objects, the conversion 
of the reference information is a simple one such as con- 
version from a DNS name to an IP address. Namely, it 
cannot only handle a single step of conversion and thus 
cannot be used for such complex name solutions that 
requires multi-stage conversion. Further, it cannot ade- 
quately handle complex reference information that is 
based on a plurality of pieces of information either. 

This means that the DNS is not suitable as the ref- 
erence mechanism used in a more sophisticated paral- 
lel distributed processing environment as mentioned be- 
fore. Namely this is because, in such a parallel distrib- 
uted processing environment, it is necessary to efficient- 
ly refer to various computation resources and various 
objects on the network by using various names and ID'S, 
but the DNS cannot handle such a reference solution. 

Respective different aspects of the invention are set 
forth in claims 1, 12, 13 and 14 hereof. 

According to a further aspect thereof, the invention 
provides a parallel distributed processing system 
wherein a plurality of operation processing nodes for ex- 
ecuting processes provided with one or more computa- 
tion objects are connected with each other through a 
network, wherein when calling and executing a compu- 
tation object or its facility provided in a first process by 
a second process, the second process obtains location 
information directly specifying the computation object or 
its facility on the network from referring means using as 



a key a name or identifier of the computation object or 
its facility, transmits this location information to the first 
process, and calls up the computation object or its facil- 
ity provided by the first process. 
$ Further, in the parallel distributed processing sys- 
tem of the present invention, preferably the referring 
means in comprised of a local referring means provided 
in a process receiving a call for a computation object or 
its facility for showing for the computation object or its 
*o facility which is called the correspondence between an 
identification number of the computation object or its fa- 
cility and an execution address of the computation ob- 
ject or its facility and a remote referring means provided 
in a process originating a call for a computation object 
'5 or its facility for showing for the computation object or 
its facility which it calls the correspondence between a 
name or identifier of the computation object or its facility 
and an identification number of the computation object 
or its facility, a process originating a call uses the name 

20 or identifier of the computation object or its facility which 
it calls as a key to obtain a corresponding identification 
number from the remote referring means and sends a 
message containing the identification number to the 
process receiving the call for the computation object or 

2S its facility, and the process receiving the call uses the 
identification number contained in the message sent 
from the process originating the call as a key to obtain 
the execution address of the computation object or its 
facility which is called from the local referring means and 

30 executes the computation object or its facility which is 
called based on the execution address. 

Further, in the parallel distributed processing sys- 
tem of the present invention, preferably a computation 
space is prescribed which is composed of a set of any 

3S of the one or a plurality of computation objects or their 
facilities and in which uniqueness of names or identifiers 
of the computaiton objects or their facilities is required 
in only the set: and the referring means gives location 
information on a computation object or its facility by us- 

•*o ing the name or identifier of the computation object or 
its facility in the computation space as a key. 

Further, in the parallel distributed processing sys- 
tem of the present invention, preferably each operation- 
processing node has a process allocating means for al- 

J 5 locating a program module at a predetermined opera- 
tion processing node and creating a process at an op- 
eration processing node receiving the allocation based 
on allocation information indicating correspondence be- 
tween location information of the program module for 

50 realizing the process and information of the operation 
processing node receiving the allocation of the program 
module and a reference information generating means 
for generating the reference information of the compu- 
tation objects or their facilities which the program mod- 

55 ule for realizing a process refers to based on the alloca- 
tion information and the reference relationship among 
the computation objects or their facilities described in 
the program module in each process. 
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Further, the parallel distributed processing system 
of the present invention preferably prescribes a compu- 
tation space which is composed of a set of any of the 
one or a plurality of computation objects or their facilities 
and in which uniqueness of names or identifiers of the $ 
computation objects or their facilities is required in only 
the set; has a plurality of reference converting means 
for converting logical reference information specifying a 
computation object or its facility in a computation space 
to either of similar logical reference information and sys- *o 
tern reference information corresponding to the location 
information or reference information of a combination of 
the same: has a conversion control means for recursive- 
ly inputting the input logical reference information and 
the logical reference information converted and gener- '5 
ated in the reference converting means to any one of 
the plurality of reference convening means based on the 
logical reference information and converting the input 
logical reference information to the system reference in- 
formation; and converts the logical reference informa- 20 
tion of the process receiving the communication to sys- 
tem reference information by the reference converting 
means through the conversion control means and per- 
forms communication among processes. 

Preferred embodiments of the present invention 2s 
were devised in consideration of the above-described 
prior art. The preferred embodiments are described in 
detail hereinbelow with reference to the accompanying 
drawings. First, however, the preferred embodiments 
and advantageous features thereof will be described in so 
outline: 

The preferred embodiments seek to provide a par- 
allel distributed processing -system and/or method 

which; 

35 

is able to perform a facility call at a high speed and 
efficiently: 

enables the load on the user for management of 
identification names of modules to be reduced: 
enables use of a variety of formats of arguments in -to 
a facility call: 

enables dynamic downloading (additional registra- 
tion) and use of any module receiving a call without 
going through troublesome work; 
can efficiently cope with a dynamic change of com- J 5 
putation module allocation: 
enables an application programmer to describe the 
reference relationship of computation modules etc. 
without depending upon the network allocation at 
the time of execution; 50 
enables management space of computation objects 
to be freely defined when managing the computa- 
tion objects network-wide: and 
can handle multi-stage name/reference solutions 
and reference by a large number of elements etc.. 55 
can efficiently perform communication among ob- 
jects, and can provide a more sophisticated parallel 
distributed processing environment by virtue of this. 



In a preferred form of implementation of the parallel 
distributed processing system of the present invention, 
as exemplified by the preferred embodiments described 
below, at the time of initialization of the system, the re- 
lationships of facility calls between modules are ana- 
lyzed. Based on the results of the analysis, facility iden- 
tification numbers of for example a numerical format are 
allocated to the facilities to be called from another mod- 
ule, and the local referring means and the remote refer- 
ring means are prepared. A facility call between mod- 
ules is performed by using the local referring means and 
the remote referring means. Further, the module receiv- 
ing the call uses the facility identification number indi- 
cated by the numerical format as a key and uses the 
local referring means to search for the execution ad- 
dress. 

As explained above, a facility call can be made at a 
high speed and efficiently. 

Also, the load on the user for management of the 
identification names of the modules can be reduced. 

Further, a variety of formats (data structures) of ar- 
guments can be used in the facility call. 

Further, any module receiving a call can be dynam- 
ically downloaded (additionally registered) and used 
without having to go through troublesome work. 

Further, by introducing the concept of computation 
space, the user can freely form management spaces for 
managing only the required computation objects in ac- 
cordance with particular purposes. As a result, the load 
on the user for management of the names and identifiers 
of the computation objects is reduced. Further, the 
number of the managed computation- objects is de- 
creased, so the time for specifying a computation object 
can be shortened. Further, erroneous access to a com- 
putation object out of the reference space can be effec- 
tively prevented. 

Further, dynamic space management of the com- 
putation objects can be easily carried out. 

Further, the allocation information concerning the 
network node by which the program module is executed 
is described outside of the program module. Therefore, 
the network node for which the program module is exe- 
cuted can be dynamically changed by only changing the 
allocation information without amendment of the de- 
scription of the program module. Namely, after the com- 
putation object space is formed, the space configuration 
thereof can be dynamically changed. 

Further, the application programmer can describe 
the reference relationship at the time of execution of the 
program module without depending upon the network 
allocation at the time of execution. 

Namely, since the allocation information at the ex- 
ecution of the program module is specified outside of 
the program module, it is sufficient so far as the user 
describes the program module by being conscious of 
only the reference relationship of the computation object 
on the program module. 

Further, on a computer network constituted by a plu- 
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rality of computer devices etc., reference information of 
the application level is converted to system reference 
information by a process of recursive and hierarchical 
conversion, therefore it is possible to suitably cope with 
also complex reference information. Further, the reso- 
lution operation can be controlled by dividing the refer- 
ence information into groups and converting it in that 
state, requesting the resolution from other reference 
resolution units, or adding additional information con- 
cerning the resolution, therefore the system reference 
information can be suitably obtained with respect to ref- 
erence information of any format. As a result, no matter 
what the topology of the computer network, the pre- 
ferred form of implementation of the present invention 
can solve the reference information and can therefore 
contribute to the provision of a more sophisticated dis- 
tributed processing system and distributed processing 
environment. 

The invention will now be further described, by way 
of illustrative and nontimiting example, with reference to 
the accompanying drawings, in which; 

Figure 1 is a conceptual view of a parallel distributed 
processing system according to a first embodiment 
of the present invention. 

Figure 2 is a view for explaining a procedure for pre- 
paring a local reference table in the parallel distrib- 
uted processing system shown in Fig. 1. 
Figure 3 is a view for explaining a procedure for pre- 
paring a remote reference table in the parallel dis- 
tributed processing system shown in Fig. 1 
Figure 4 is a viewfor explaining the processing in a 
network node originating a call when calling a facil- 
ity in the parallel distributed processing system 
shown in Fig. 1 . 

Figure 5 is a view for explaining the processing in a 
network node receiving a call when calling a facility 
in the parallel distributed processing system shown 
in Fig. 1. 

Figure 6A is a timing chart of the parallel distributed 
processing system shown in Fig. 1 where complete 
synchronous message transmission is carried out. 
Figure 6B is a timing chart of the parallel distributed 
processing system shown in Fig. 1 where complete 
synchronous message transmission is carried out. 
Figure 7A is a timing chart of the parallel distributed 
processing system shown in Fig. 1 in the case of 
asynchronous message transmission. 
Figure 7B is a timing chart of the parallel distributed 
processing system shown in Fig. 1 in the case of 
asynchronous message transmission. 
Figure 8 is a conceptual view of the parallel distrib- 
uted processing system. 

Figure 9 in a conceptual view of a parallel distribut- 
ed processing system according to a second em- 
bodiment of the present embodiment. 
Figure 10 is a view of the configuration of the par- 
allel distributed processing system according to the 



second embodiment of the present invention. 
Figure 11 is a view for explaining a reference infor- 
mation of the parallel distributed processing system 
shown in Fig. 10. 

Figure 12 is a view for explaining a local reference 
tables. 

Figure 13 is a view for explaining a remote refer- 
ence tables. 

Figure 14 is a view for explaining a process refer- 
ence table. 

Figure 15 is a view for explaining a domain refer- 
ence table. 

Figure 16 is a view for explaining a domain name 
hash table. 

Figure 1 7 is a view for explaining a function/process 
name hash table. 

Figure 1 8 is a view for explaining a remote function 
ID hash table 

Figure 1 9 is a view for explaining a space configu- 
ration for generating the reference information. 
Figure 20 is a view for explaining a process in a ref- 
erence generation unit of a management process. 
Figure 21 is a view for explaining an initialization 
processing of a domain reference table on a proc- 
ess P1 shown in Fig. 19. 

Figure 22 in a view for explaining the initialization 
processing of the domain reference table on the 
process P1 shown in Fig. 19. 
Figure 23 is a view for explaining the initialization 
processing of a local function reference table on the 
process P1 shown in Fig. 19. 
Figure 24A is a view for explaining the initialization 
processing of a local function reference table on the 
process P1 shown in Fig. 19. 
Figure 24B is a view for explaining the initialization 
processing of a local function reference table on the 
process P1 shown in Fig. 19. 
Figure 25 is a view for explaining the initialization 
processing of a local function reference table on a 
process P2 shown in Fig. 19. 
Figure 26A is a view for explaining the initialization 
processing of a local function reference table on the 
process P2 shown in Fig. 19. 
Figure 26B is a view for explaining the initialization 
processing of a local function reference table on the 
process P2 shown in Fig. 19. 
Figure 27 is a view for explaining the initialization 
processing of a process reference table on the proc- 
ess P1 shown in Fig. 19. 

Figure 29 is a view for explaining the initialization 

processing of the process reference table on the 

process Pi shown in Fig. 19. 

Figure 29 is a view for explaining the initialisation 

processing of a remote function reference table on 

the process P1 shown in Fig. 19. 

Figure 30 is a view for explaining the initialization 

processing of the remote function reference table - 

on the process P1 shown in Fig. 19. 
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Figure 31 is a view for explaining the initialization 
processing of the. remote function reference table 
on the process P2 shown in Fig. 19. 
Figure 32 is a view for explaining the initialization 
processing of the remote function reference table $ 
on the process P2 shown in Fig. 19. 
Figure 33 is a view of the configuration of the par- 
allel distributed processing system according to a 
third embodiment of the present embodiment. 
Figure 34 is a view of the content of description of io 
a progra_m module processed in the parallel distrib- 
uted processing system shown in Fig. 33. 
Figure 35 is a view for explaining a process alloca- 
tion unit shown in Fig. 33. 

Figure 36 is a view for explaining a computation '5 
space and domain formed in the parallel distributed 
processing system shown in Fig. 33. 
Figure 37 is a view for explaining a process ar- 
ranged in each network node of the parallel distrib- 
uted processing system shown in Fig. 33. 20 
Figure 38 is a flowchart of the processings in a mas- 
ter management process C and slave management 
processes A and B. 

Figure 39 is a view of the configuration of a process 
allocation table. 2s 
Figure 40 is a concrete example of the process al- 
location table of a network node C. 
Figure 41 is a concrete example of the process al- 
location table of a network node A. 
Figure 42 is a concrete example of the process al- 30 
location table of a network node B. 
Figure 43 is a view of the configuration of a port 
number assignment table. 

Figure 44 is a view of the configuration of the port 
number assignment table. 3$ 
Figure 45 is a flowchart of the processing of a gen- 
eral process. 

Figure 45 is a flowchart for explaining an import 
processing. 

Figure 47 is a view for explaining the import 4 o 
processing. 

Figure 48 is a view for explaining the import 
processing. 

Figure 49A is a view for explaining the import 
processing. ^5 
Figure 49B is a view for explaining the import 
processing. 

Figure 50 is a view for explaining an export process- 
ing. 

Figure 51 is a view for explaining the export so 
processing. 

Figure 52A in a view for explaining the export 
processing. 

Figure 52B is a view for explaining the export 
processing. S5 
Figure 53 is a view for explaining the export 
processing. 

Figure 54 is a view for explaining the export 



processing. 

Figure 55 is a block diagram of the configuration of 
a reference conversion device of a fourth embodi- 
ment of the present invention. 
Figure 56 is a flowchart of the flow of the processing 
of a control unit shown in Fig. 55. 
Figure 57A is a view of an example of operation of 
the reference conversion device shown in Fig. 55. 
Figure 57B is a view of an example of operation of 
the reference conversion device shown in Fig. 55. 
Figure 57C is a view of an example of operation of 
the reference conversion device shown in Fig. 55. 
Figure 53 is a block diagram of the configuration of 
a reference resolution unit in the distributed 
processing system of a fifth embodiment of the 
present invention. 

Figure 59 is a view of an example of operation of 
the reference solution unit shown in Fig. 53. 
Figure 60 is a view of a concrete embodiment of the 
reference resolution unit shown in Fig. 58. 

Below, an explanation will be made of a parallel dis- 
tributed processing system according to an embodiment 
of the present invention. 

The parallel distributed processing system accord- 
ing to the present embodiment supports the execution 
environment of, for example, an ISM, enhances the di- 
alog between facility modules and the flexibility of the 
inter-module structure, and thereby enables network- 
wide multimedia parallel processing. 

First Embodiment 

Figure 1 isa conceptual view of a parallel distributed 
processing system 1 embodying the present embodi- 
ment. 

As shown in Fig. 1, in the parallel distributed 
processing system 1 , network nodes 3 and 4 communi- 
cate via a network 2 to perform predetermined process- 
ing. As the network nodes 3 and 4, for example use may 
be made of host ccomputers. In the network node 3 : pro- 
grams such as a management process 30 and compu- 
tation objects 300, 301. and 302 operate. Here, a com- 
putation object executing application program is com- 
prised of a definition of a function, variable, class and 
method which define an operation of said computation 
object, and the operation of an instance generated 
based on the class is described in a definition of a func- 
tion and method, etc. 

The management process 30 has a local reference 
table 306 and a remote reference table 310 as will -be 
explained later. 

The local reference table 306 is a table indicating 
the correspondence of the facility identification codes 
and the addresses at which the facilities thereof are 
stored. The local reference table 306 is used for obtain- 
ing the address in the computation objects 300 to 302 
of the facility which is called from the facility identifica- 
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tion code contained in the message input from the net- 
work node 4 when a facility call is made from the network 
node 4 to the network node 3. 

The remote reference table 310 is a table showing 
■ the correspondence of the addresses as the facility s 
identifiers at which the facility identification codes are 
stored and the facility identification codes. The remote 
reference table 310 is used for obtaining the facility iden- 
tification code of a facility from the address referred in 
the computation objects 300 to 302 when a facility call io 
is made from th"e network node 3 to the network node 4. 

Further, in the network node 4. programs such as a 
management process 40 and computation objects 400, 
401. and 402 operate. 

The computation objects 400 to 402 of the network is 
node 4, the local reference table 406, and the remote 
reference table 410 have the same meanings and facil- 
ities as those of the computation objects 300 to 302 of 
the network node 3, the local reference table 306, and 
the remote reference table 310. 20 

In the parallel distributed processing system 1 
shown in Fig. 1 , the allocation of the computation ob- 
jects, that is, in which network node or processor which 
computation object is allocated, is in principle analyzed 
(designated) only one time at the time of initializing the 25 
system. Accordingly, at the time of execution of the sys- 
tem, each computation object already knows the loca- 
tion of the computation object which receives a call 
when calling the facility of another class. Note that the 
allocation of the computation objects is sometimes des- 30 
ignated at the time of execution of the system. 

Note that, -the network 2 may have other network 
nodes connected to it too in addition to the network 
nodes 3 and 4. 

Below, an explanation will be made of the process- 35 
ing in the parallel distributed processing system 1 taking 
as an example the case where a facility provided in the 
computation object 402 of the network node 4, more 
specifically a method, is called when performing the 
processing of a method call in object-oriented compu- *o 
tation. that is, the processing of the computation object 
302 at the network node 3. 

In the processing of the method call in the object- 
oriented computation, the method in the computation 
object is for example a functional procedure or other fa- 4 $ 
cility. 

(Preparation of Local Reference Table and Remote 
Reference Table) 

. 50 

In the parallel distributed processing system 1, 
where for example the computation object 302 is newly 
added (registered) in the network node 3 shown in Fig. 
3, and the computation object 302 calls the facility of the 
computation object 402 of the network node 4 shown in 55 
Fig. 2, the remote reference table 310 of the manage- 
ment process 30 shown in Fig. 3 and the local reference 
table 406 of the management process 40 shown in Fig. 



2 are prepared or updated as shown below. 

First, an explanation will be made of the preparation 
of the local reference table. 

The network nodes 3 and 4 prepare the local refer-, 
ence tables 406 and 306 shown in Fig. 2 and Fig. 3 for 
the classes, the methods, the instances and the func- 
tions defined by these computation objects at for exam- 
ple the time of initializing the system. 

Here, the local reference table 306 is comprised by 
a local class table 303, a local function table 304, a local 
instance table 305, and a local method table 330. 

Further, the local reference table 406 is comprised 
by a local class table 403, a local function table 404, a 
local instance table 405, and a local method table 430. 

Here, the local method tables 330 and 430 show 
the correspondence between the 'method ID" of the 
method and the "method address", that is, the execution 
address of the method. 

Note that, the relationships of calls of the methods 
between the network nodes 3 and 4 and other network 
nodes are analyzed in advance at the time of initializing 
the system and when the computation object 402 is 
newly added. At the stage for preparing the remote ref- 
erence table 310 and the local reference table 406, a 
"method ID (facility identification code)" and a "class ID* 
are automatically allocated to the method and the class 
of the computation object 402, respectively. 

Next, an explanation will be made of the preparation 
of the remote reference table. 

For example, where the computation object 302 
shown in Fig. 1 calls method provided in the computa- 
tion object 402. as shown in Fig. 3, the network node 3 
outputs an inquiry 312 to the network node 4 using the 
computation object 402 and method name from a dis- 
patcher 311 of the computation object 302. 

When receiving this inquiry 312. the network node 
4 refers to the local reference table 406, inserts the 
."class ID" and "method ID" assigned to the computation 
object 402 and the method of the computation object 
402 in an answer message 315 shown in Fig. 3, and 
outputs the same to the network node 3 via the network 
2 shown in Fig. 1. The answer message 315 is output 
to the dispatcher 311 via the message queues (waiting- 
matrixes) 314 and 31 3 in the network node 3. 

Next, the dispatcher 31 1 perlorms the registration 
processing and prepares a remote class table 307 and 
a remote method table 331 by using the "class ID' and 
'method ID" contained in the answer message 315. 

Here, the remote reference table 310 is comprised 
by a remote class table 307. a remote function table 303, 
and a remote instance table 309, and a remote method 
table 331. 

The local reference table and the remote reference 
table are prepared as an initialization process when the 
network node 3 and the network node 4 are connected 
via the network 2 or are automatically prepared only one 
time when a new computation object is dynamically 
downloaded (additionally registered) in the network 
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nodes 3 and 4. 
[Facility Call Processing] 

Below, an explanation will be made of a case where 
a facility call is made from the network node 3 to the 
network node 4 by using the remote reference table 31 0 
and the local reference table 406 prepared as men- 
tioned before. Here, the explanation will be made of the 
case of a so-called complete synchronous message 
transmission jn which the computation object originating 
the call calls the facility, then stops its processing until 
it receives as input the result of execution of the function 
from the computation object receiving the call. 

For example, when a method of the computation ob- 
ject 402 provided in the network node 4 is called when 
processing is carried out in the computation object 302 
of the network node 3 shown in Fig. 4 ("AT shown in 
Fig. 6A), the dispatcher 311 of the computation object 
302 converts the data structure of the arguments con- 
tained in the message to the data expression of the log- 
ical format ("81* shown in Fig. 6A). In the conversion, it 
uses a conversion program to analyze the data structure 
of the internal expressed data in the computation object 
302 and converting it to a data expression of the logical 
format. At this time, the data expression of the logical 
format adopts a description syntax enabling a complex 
data structure such as a list structure to be expressed 
by combining simple data. For this reason, a variety of 
data structures can be used as the arguments. For this 
reason, the range of the type of methods and functions 
which can be covered by the facility call can be expand- 
ed. - * 

Next, in the dispatcher 311. the above-mentioned 
addresses identifying the method and the class of the 
computation object 402 are output to the remote method 
table 331 and the remote instance table 309. respec- 
tively. By this, the function identification code is referred 
to and the "method ID" and "class ID" corresponding'to 
these addresses are searched for ("B2" shown in Fig. 
6A). The retrieved "method ID" and "class ID" are reg- 
istered in the transmission queue 31 3 in the dispatcher 
31 1 {"B3" shown in Fig. 6A). 

Next, the "method ID" and "class ID" registered in 
the transmission queue 31 3 are registered in the trans- 
mission queue 314 of the management process 30 ("C1 " 
shown in Fig. 6A). In the management process 30. the 
message header of the message 320 is prepared by us- 
ing the "method ID" and "class ID", and the message 
containing this message header is transmitted to the re- 
ception queue 414 of the management process 40 of 
the network node 4 shown in Fig. 5 via the network ("C2" 
shown in Fig. 6A). The network node 3 stops the 
processing of the program being executed until it re- 
ceives as input the result of processing of the method 
from the network node 4. 

In this way, by using the message queues 314 and 
414. the messages are accumulated in the queues and 



then processed by the method (facility). For this reason, 
even if the facility call requests with respect to a certain 
specific computation object compete by these queues, 
they are processed in order or according to the priority 

s order without a loss of the call request. 

When receiving the message from the network 
node 3 ("D1" shown in Fig. 6A). the management proc- 
ess 40 of the network node 4 registers that message in 
the reception queue 414 shown in Fig. 5 ("D2" shown in 

10 Fig. 6B). Next, the signal is output from the management 
process 40 to the dispatcher 411 ("D3" and "ET shown 
in Fig. 6B). 

Next, the dispatcher 411 refers to the local instance 
table 405 shown in Fig. 5 by using the "class ID" as the 

'5 key and calls the predetermined instance ("E2" shown 
in Fig. 6B). The dispatcher 411 executes the next 
processing until a state where the predetermined in- 
stance can be called is exhibited ("E3" shown in Fig. 6B). 
At this time, the commmication manager provided 

20 in the message queue 414 uses the "class ID' contained 
in the message 320 as a key to refer to the local instance 
table 405 and obtain the address of the instance of the 
computation object 402. By this, the instance of the com- 
putation object 402 is determined. Here, "instance" 

25 means an facility showing a state where the processing 
of the class is actually carried out. 

Note that the instance in the computation object 402 
is generated, as shown in Fig. 5, by referring to the "ad- 
dress of the instance generation function" stored corre- 

30 sponding to the computation object 402 in the local class 
table 403. executing the instance generation function 
421 existing at the "address of the instance generation 
function", and registering the instance generated by this 
in the local instance table 405, 

35 Next, the dispatcher 4 1 1 converts the data structure 
of the message to the same data structure as that used 
in the original computation object 302 ("E4" shown in 
Fig. 6B). 

Next, based on the "method ID" and the "class ID" 

•io contained in the received message 320. the "method ad- 
dress" of the function for the facility which is called is 
obtained by using the local method table 430. At this 
time, the "method ID" and "class ID" are indicated by 
numerals, therefore are collated at a high speed. 

•*5 in the network node 4. the processing in the com- 
putation object 402 is carried out by using the "method 
address" of the method obtained in the dispatcher 411, 
the instance determined by the communication manag- 
er, and the local expression data 420 obtained by con- 

50 version in the dispatcher 411 ("F" shown in Fig. 6B). 

The result of this processing is sent to the process- 
ing of the computation object 302 of the network node 
3 originating the call shown in Fig. 3 by performing the 
processing of "E5", "D4". and "D5" shown in Fig. 6B and 

55 "C3", "C4\ "C5". "B4", "B5". and "A2" shown in Fig. 6A 
as return values. By this, the facility call processing is 
completed. 

Here. "E5" means the conversion of the data ex- 
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pression of the return value, "D4" means the registration 
of the return value in the transmission queue. "D5' 
means the transmission of the message. "C3" means 
the reception of the message. *C4" means the registra- 
tion to the reception queue, "C5" means the signal trans- 
mission to the class 302. "B4" means the signal recep- 
tion, "85" means the conversion of the data expression 
of the return value, and "A2* means the continued exe- 
cution of the next processing. 

The network node 3 resumes the processing of the 
stopped program when the result of the processing of 
the method is input from the network node 4. 

As explained above, according to the parallel dis- 
tributed processing system 1 1 the callup of a facility such 
as a method is made by using the local reference table 
and remote reference table showing the relationship of 
the facility calls among the computation objects and the 
relationship of the facility identification codes etc. indi- 
cated by the numerals for the facility to be called, there- 
fore the address of the execution unit of the facility which 
is called can be specified at a high speed. 

Further, according to the parallel distributed 
processing system 1 , the system automatically gives a 
unique facility identification code for the address of the 
computation object and method for which the facility call 
is made. For this reason, it is not necessary for the user 
(programmer) to expend much tabor for avoiding the col- 
lision of the addresses of the computation objects and 
methods, so the load on the user is reduced. 

Further, according to the parallel distributed 
processing system 1 . the local reference table and the 
remote reference table are dynamically prepared in aa- 
cordance with the addition of a computation object, 
therefore it becomes possible to dynamically download 
the necessary computation object at the designated net: 
work node. 

Further, according to the parallel distributed 
processing system 1, it is not necessary to define the 
arguments of the facility call and the format of the result 
of the processing (return value) at the time of compila- 
tion, therefore the flexibility of the facility call can be en- 
hanced. 

In order to efficiently use the processors and other 
resources of the system, there is a programming system 
for automatically dividing a program into smaller parallel 
distributed execution units at a level that is not apparent 
to the user (programmer). As opposed to this, in the par- 
allel distributed processing system 1. the user divides 
the computation object into parallel distributed modules 
in units of large facilities and can clearly designate the 
allocation of the parallel distributed modules, so can 
construct a large application system. 

Next, an explanation will be made of another exam- 
ple of the case where a facility call is made from the 
network node 3 to the network node 4. Here, the expla- 
nation will be made of the case of so-called synchronous 
message transmission where a computation object orig- 
inating a call calls a facility, then does not receive as 



input the result of execution of the facility from the com- 
putation object receiving the call. 

Figures 7A and 7B are timing charts of the case of 
the asynchronous message transmission. 

s Namely, if the method of the computation object 402 
provided in the network node 4 is called by asynchro- 
nous message transmission when the processing is car- 
ried out in the computation object 302 of the network 
node 3 shown in Fig. 4 (*G1* shown in Fig. 7 A), the 

io processings of 'HI". "H2 - , *H3" , •IV. "12* . and "G2" 
shown in Fig. 7A are executed in the computation object 
302, dispatcher 31 1 , and management process 30 of the 
network node 3. 

The processings of "HT. "H2 ■ . 'A3 \ "IV. and "12* 

is shown in Fig. 7A correspond to the processings of "B1 ", 
"B2\ "B3'. "CI", and 'C2' shown in Fig. 6A mentioned 
before. 

However, in the example shown in Figs. 7A and 7B. 
the computation object 302 does not wait for the result 

20 of processing of the facility by the computation object 
402 of the network node 4 which was called, but exe- 
cutes the next processing 'G2' after the processing '12'. 

On the other hand, in the network node 4. the man- 
agement process 40. the dispatcher 41 1 . and the com- 

25 putation object 402 execute the processing of "J1 ", "J2"*. 
■J3". "K1 '. 'K2*. "K3\ 'K4'. and V shown in Fig. 7B. 
Here, the processing of "jr. "J2\ \J3\ 'K1\ "K2 ' . 
"K3\ "K4 • : and V shown in Fig. 7B are basically the 
same as the processing of the "D1". "D2'. - D3' . 'ET . 

30 "E2", "E3" , "E4 " . and "F" shown in Fig. 6B. but the re- 
sult of processing of 'F' is not transmitted to the network 
node 3. 

Next, the concept of the parallel distributed 
processing system according to the present embodi- 
es ment explained above will be summarized. 

As shown in Fig. 3. the network node 96 remote- 
calls (refers) a method 105a of the object of the desti- 
nation the call of the network node 99 in the object orig- 
inating the call. At this time, the remote instance refer- 
jo ence and remote method reference are performed. 
Namely the entity of the object for calling is specified by 
specifying both the instance and method. 

Note that, in Fig. 8 the specification of the instance 
is illustrated. 

That is. the remote instance reference table 102 is 
refereed by using a key described in the instance 101b 
of the object originating the call 101 a, and then the re- 
mote instance ID r #01 23J is obtained. Further the proc- 
ess reference table 107 is refereed by the pointer de- 

so scribed in the process reference of the remote instance 
reference table 102. and them the process lD r #0055j 
to which the method receiving the call belongs and the 
network address r 1 92. 1 63. 1 . 1 J of the network node 99 
on which the process operates are obtained. Further the 

55 argument 1 0 1 c 1 is indicated by the pointer described in 
the argument 1 01 d. and then the argument 1 01 c2 which 
is the logical conversion of the data structure of the ar- 
gument lOld is obtained. 
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And then, the message including the remote in- 
stance ID r #0123j, process ID r #0055J and the argu- 
ment 101c2 are transmitted to the network node 99 via 
network based on the network address r 192.168 1 .1 J. 

When receiving the message, the network node 99 
specifies the local instance reference table 103 by the 
process ID r #0055j, and obtains the local instance ID 
of the object receiving the call 108 by using the remote 
instance ID r #0123j as a key. The object receiving the 
call is specified by the local instance ID. 

The remote method reference table and local meth- 
od reference table are referred by using a key described 
in the method 1 01c of the object originating the call 1 01 a 
in the same way described above, the local method ID 
is obtained. The method 105A of the object receiving 
the call 105 A is specified by the local method ID. The 
entity of the method 106 is indicated by the pointer de- 
scribed in the method 105a. 

On the other hand, the argument 101c which is in- 
putted by the network node 99 via network is converted 
reversely from the logical-format of the data structure, 
and then becomes to be argument 1 01 d . The argument 
101 cl is indicated by the pointer by the argument 105b 
corresponding to the method 105a. 

And then, the entity of the method 106 is provided 
with the method 105a, and then the entity of the method 
106 operates. 

The present invention is not limited to the above em- 
bodiments. For example, in the above embodiments, a 
case where the present invention was applied to a callup 
of a method in object directed programming was exem- 
plified, but the present invention can be applied to for 
example a remote -function call too. A remote function 
call calls a function (facility) existing in a module such 
as a remote application. 

Further, the present invention can also be applied 
to a case where an internal variable of a certain compu- 
tation ob|ect is used as the shared variable (global var- 
iable) among groups of distributed modules (computa- 
tion objects). In this case, as the facilities of the compu- 
tation objects containing the variable which becomes 
the common variable, the facilities of reading, change, 
etc. of the common variable are programmed. At this 
time, the access to the common variable is all made by 
a facility call to the computation object containing the 
variable thereof, therefore exclusive control of the ac- 
cess to the common variable thereof is naturally real- 
ized. Namely, the common variable is held in the in- 
stance. For reading and rewriting the common variable, 
it is necessary to call the facility of the instance. By doing 
this, the instance acts as the common variable itself. 

At this time, the common variable is defined as a 
class of the computation object, and the processing for 
the common variable is defined as the method (func- 
tion). 

Further, in the above embodiment, in the network 
node 3 acting as the module originating the call, the ad- 
dress at which the facility identification code was stored 



as a facility identifier was exemplified, but if the facility 
name indicated by characters or a number which is valid 
only in the module originating the call is used as the fa- 
cility identifier, further higher speed facility call process- 
s ing can be carried out. 

Further, it is also possible to determine the order of 
output of messages based on a predetermined priority 
order in addition to output of a queue accumulating mes- 
sages input or output between the network nodes 3 and 
io 4 in the order of the input. Namely, it is also possible to 
give a priority order to a facility call. This priority order 
can be determined as a system in advance or can be 
contained in the message as attribute information. 
Further, at the message transmission side, it is also 
is possible to include the transmission time in the message 
and determine the time of execution of the facility spec- 
ified by the message based on the time at the reception 
side. Further, the queue receiving the message can be 
provided for every instance (class) or can be shared by 
a plurality of instances (classes). Further, within the 
scope of the invention, it is also possible for the network 
node 3 to continue its processing until the result of 
processing from the network node 4 becomes neces- 
sary after outputting the message to the network node 
4 and to stop the processing of the program being exe- 
cuted only in a case where the result of processing has 
not yet been received at the point of time when it be- 
comes necessary. 

At this time, it is also possible for the network node 
3 to automatically activate an error processing program 
when the result of processing for the message which 
has been already sent cannot be obtained from the net- 
work node 4 even if a set time elapses. 

Further, within the scope of the invention, it is also 
possible for the network node 3 not to request the result 
of processing from the network node 4 after outputting 
the message to the network node 4. 

Second Embodiment 

Figure 9 is a conceptual view of a parallel distributed 
processing system 1 according to the present embodi- 
ment. 

As shown in Fig. 9, in the parallel distributed 
processing system 1. the network nodes 3, 4. and 5 
communicate with each other via a network 2 and per- 
form predetermined processing. As the network nodes 
3, 4. and 5 : for example, personal computers and work 
stations are used. Further, as the network 2, for example 
an Ethernet or CATV is used. 

In the network nodes 3, 4, and 5, computation ob- 
jects, for executing a program, for example, functions, 
instances, classes, methods, global variables, and files 
exist. Here, a global variable is realized with an instance 
of a class of global variable. 

For example, the computation objects 6 to 9 exist 
in the network node 3. 
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[Space Management System] 

In the network nodes 3, 4, and 5, partial spaces 
comprised by a plurality of computation objects in the 
same network node exist. For example, in the network 
node 3, a partial space 10 is comprised by the compu- 
tation objtets 6 and 7, and a partial space 11 is com- 
prised by the computation objects 8 and 9. Here, a "par- 
tial space" is a process generated by a management 
process, that is. a computation module, provided at the 
network node. - ' 

Further, a computation space may be comprised by 
partial spaces and/or computation objects existing in dif- 
ferent network nodes. In the present embodiment, a 
computation space 15 is comprised by the partial spac- 
es 1 0. 1 2. and 1 4, and a computation space 16 is com- 
posed by the partial spaces 11 and 13. Note that, it is 
also possible to comprise a computation space so as to 
contain partial spaces and/or computation objects exist- 
ing in the same network node. Here, a "computation 
space" is a region in which information concerning the 
allocation of computation objects and partial spaces ex- 
isting in the space is clarified. 

Further, in the parallel distributed processing sys- 
tem 1 , the allocation of the computation objects and the 
partial spaces is determined according to information 
given in advance, inquiries between the partial spaces, 
etc. The allocation of the computation objects and the 
partial spaces is managed by space management units 
17 and 18. 

In the parallel distributed processing system 1, as 
shown in Fig. 9. a space management unit 17 is provid- 
ed in the network node 4 corresponding to the compu- 
tation space 15, and a space management unit 18 is 
provided in the network node 4 corresponding to the 
computation space 15. 

Note that, it is also possible to provide a space man- 
agement unit for every partial space or provide the same 
for every computation object. 

Further, in the parallel distributed processing sys- 
tem 1 . there is a management process for every partial 
space. The reference information is managed by this 
management process. Here, a space expressed by the 
reference information will be referred to as a 'reference 
space*. In an API (application programming interface), 
the processes and computation objects are identified 
and managed on this reference space. An actually ex- 
isting space which is referred to will be referred to as a 
"real space" with respect to this reference space. 

Here, "reference" means that these processes and 
computation objects are logically indicated. 

In the present embodiment, the concept of a domain 
for forming any reference set in this reference space is 
further introduced. This domain is used when dividing a 
computation space into a plurality of parts for manage- 
ment. The domain is not shown in Fig. 9 and will be ex- 
plained in detail later by referring to Fig. 19. 

A process (partial space) and computation object 



may belong to a plurality of domains. It is also allowed 
that the domains take a hierarchical structure. In this 
case, a contained domain will be referred to as a sub- 
domain, and a containing domain will be referred to as 

s a super-domain. By the introduction of domains, space 
(service space, application space, personal space, 
community, etc.) can be flexibly formed network-wide. 

In the present embodiment, a name is further given 
to each reference, and an identification space of the 

io processes, computation objects, and domains by 
names is provided. This identification space will be re- 
ferred to as a "name space". From the API . it is possible 
to identify and manage the processes, computation ob- 
jects, and domains even on this name space. Further, 

is the uniqueness of names is guaranteed in only each 
name space. 

Accordingly, the collision of names is allowed be- 
tween computation objects belonging to different do- 
mains and thus the degree of freedom of the names use- 

20 able in each domain is raised. 

Below, an explanation will be made of the reference 
information used in the parallel distributed processing 
system according to the present embodiment. 

Figure 10 is a view of the configuration of a parallel 

2S distributed processing system 31 according to the 
present embodiment. 

As shown in Fig. 10, in the parallel distributed 
processing system 31 , network nodes 33 to 35 are con- 
nected via a network 32. 

30 In the network node 33. a process (partial space) 
36 generated by the management process 38 exists. 

The management process 38 has a reference hold- 
ing unit 39 and a reference generation unit 40. Here, the 
reference holding unit 39 stores reference information 

35 identifying the processes, computation objects, and do- 
mains for specifying their locations and controlling and 
managing the same. 

In the following example, a case is shown where 
functions allocated distributedly throughout a network 

40 are referred to. The reference information is comprised 
of four reference tables and three search use hash ta- 
bles as shown in Fig. 11 . As the reference tables, there 
are a local function reference table 50 shown in Fig. 1 2,- 
a remote function reference table 51 shown in Fig. 13, 

■*s a process reference table 52 shown in Fig. 14, and a 
domain reference table 53 shown in Fig. 15. Note that 
Fig. 12 and Fig. 13 also describe reference tables for 
the classes, instances, and methods in addition to func- 
tions. 

so Further the search-use hash tables are hash tables 
between the names and identifiers, include a domain 
name hash table 54 shown in Fig. 1 6, a function/process 
name hash table 55 shown in Fig. 1 1, and a remote func- 
tion ID hash table 55 shown in Fig. 18. 

55 Note that, as the network node 34, there is a proc- 
ess (partial space) 37 generated by the management 
process 41. This is conceptually the same as the net- 
work node 33. 
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[local Function Reference Table] 

Figure 12 is a view for explaining the local function 
reference table 50. 

The local function reference table 50 is a local ref- 
erence table of the functions existing in a partial space 
(process). 

As shown in Fig. 12. the local function reference ta- 
ble 50 has five fields of a print name 61 , a local function 
ID 62, a domain reference list 63, a remote function ID 
64, and an in-domain name 65. 

The print name 61 is the function name on the pro- 
gram source code. If it is a class provided for example 
by C++, the class name thereof is inserted. 

The local function ID 62 is a function identifier in the 
network node used on the system. Access is actually 
carried out by using this function identifier 

The domain reference list 63 is a pointer list to the 
domain reference tables to which the function thereof 
belongs. The domain reference list 63 is indicated by 
data having a list structure as shown in Fig. 12. This is 
for enabling expression when a function belongs to a 
plurality of domains. 

The remote function ID 64 is an external identifier 
of the function. A function is designated and called from 
outside of the network node by using this external iden- 
tifier. 

The in-domain name 65 is a name inside the domain 
on the name space. This name is used for designating 
and calling a function. 

[Remote Function Reference Table] 

Figure 13 is a view for explaining a remote function 
reference table 51. 

The remote function reference table 51 is a remote 
reference table of functions existing in partial space 
(process). 

As shown in Fig. 13, the remote function reference 
table 51 has five fields of a print name 71, a process 
reference 72, a remote function ID 73. a domain refer- 
ence list 74. and an in-domain name 75. 

The meanings of the print name 71, the remote 
function ID 73, the domain reference list 74. and the in- 
domain name 75 are the same as those of the case of 
the local function reference table 50 mentioned before. 

The process reference 72 is information concerning 
the location of the process in which the remote function 
thereof exists and for example indicates a pointer to the 
process reference table 52. . 

[Process Reference Table] 

Figure 14 is a view for explaining a process refer- 
ence table 52. 

The process reference table 52 is a table for storing 
information indicating the locations of partial spaces 
(processes). 



As shown in Fig. 14. the process reference table 52 
has four fields of a network node reference 8 1 . a network 
port ID 62, a domain reference list 83. and an in-domain 
name 84. 

5 Here, the meanings of the domain reference list 83 
and the in-domain name 84 are the same as those men- 
tioned before. 

The network node reference 81 indicated reference 
information concerning the network node in which a 
io process exists and has four of a network node mame 
85. a reference list 88, a communication media 65, and 
a network address 87. 

The network node node 85 indicates the name of 
the network node on the name space. 
i$ The communication media 86 indicates the commu- 
nication means and indicates for example Ethernet or 
ATM. 

The network address 87 indicates the network ad- 
dress of the node in accordance with the communication 
means and indicates for example an IP address and 
DNS (domain name system) name. 

The network port ID 82 indicates the management 
process port number in each network node used on the 
parallel distributed processing system 31 . Access is car- 
ried out by specifying a process by the network address 
and this management process port number. 

There are network nodes provided with a plurality 
of communication media (access facilities). For exam- 
ple, among the network nodes, there is one to which ac- 
cess by both of the Ethernet and ATM is possible. Usu- 
ally, in such a case, network addresses are individually 
assigned to the access facilities. - 

Accordingly, in the process reference table 52. by 
expressing the correspondence between the network 
names and the network addresses by using the refer- 
ence list 53 of the list structure as shown in Fig. 14. the 
correspondence between one network node name 85 
and a plurality of network addresses 87 is expressed in 
accordance with the types of the communication media 
86. 

In the example shown in Fig. 14, a process with an 
in-domain name "NT has the network port ID 82 
"#10000* and the network node name 85 "node A". Fur- 
ther, "node A" is provided with the Ethernet and ATM as 
the communication media 36. The network addresses 
87 are '111.111.111 .111 " and "111.111. 111. 11 2', 

[Domain Reference Table] 

Figure 15 is a view for explaining a domain refer- 
ence table 53. 

The domain reference table 53 is a table for man- 
aging the reference information of the domains. As men- 
tioned before, the domains can adopt a hierarchical 
structure and so one domain can contain another do- 
main. 

As shown in Fig. 15. the domain reference table 53 
has a domain element reference list 91 . a super-domain 
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reference list 92. and a domain name 93. 

The domain element reference list 91 indicates the 
pointer to the function reference and sub-domain refer- 
ence which are constituent elements of a domain. 

The super-domain reference list 92 indicates a 
pointer list to the super-domain (higher domain) of the 
domain. Here, the reason for the use of the list structure 
for the super-domain reference is that one domain 
sometimes has a plurality of super-domains. 

The domain name 93 indicates the domain name 
on .the name space. 

[Domain Name Hash Table] 

Figure 16 is a view for explaining a domain name 
hash table 54. 

The domain name hash table 54 is constructed 
based on the domain names given by individual name 
spaces and enables a search of the domain reference 
table by name. 

As shown in Fig. 16, the domain name hash table 
54 has two fields of a domain name 1 00 one a reference 
list 101. 

The domain name 100 indicates the domain name 
on the name space. The reference list 101 indicates the 
pointer list to the domain reference table. 

In the example shown in Fig. 16, when the domain 
name hash table 54 is used using the domain name '01 r 
as a key, the address of the domain name "DT of the 
domain reference table 53 is obtained. 

[Function/Process Name Hash Table] 

Figure 1 7 is a view for explaining a function/process 
name hash table 55. 

The function/process name hash table 55 is con- 
structed based on the in-domain names given by indi- 
vidual name spaces and enables a search of the func- 
tion reference table and the process reference table by 
name. 

As shown in Fig. 17, the function/process name 
hasb table 55 has two fields of an in-domain name 110 
and a reference list ill. 

As the in-domain name 110, either of the local func- 
tion name in the domain on the name space, the remote 
function name, and the process name is indicated. 

The reference list 111 is a pointer list with respect 
to either of the local function reference table 50. the re- 
mote function reference table 51, and the process ref- 
erence table 52. 

For example, in Fig. 17. when the local function 
name hash table of the function/process name hash ta- 
ble 55 is used using the function name "A' of the in-do- 
main name nOasa key, the reference list m gives the 
address of the in-domain name "A" in the local function 
reference table 50. As the function/process name hash 
table 55. other than the local function name bash table, 
there are a remote function name hash table and a proc- 



ess name hash table. 

[Remote Function ID Hash Table] 

5 Figure 1 8 is a view for explaining a remote function 
ID hash table 56. 

The remote function ID hash table 56 is constructed 
based on the remote function IDs and enables a search 
of the local function reference table by the remote fuhc- 

io tion IDs. 

As shown in Fig. 18. the remote function ID hash 
table 56 has two fields of a remote function ID 120 and 
a local function reference 121. 

The local fuction reference 1 21 indicates the pointer 
is to the local function reference in the local function ref- 
erence table corresponding to a remote function ID. 

For example, in Fig. 18, when the remote function 
ID hash table 56 is used using •#000001 ' of the remote 
function ID 120 as a key, the address of the position 
20 where the remote function ID 64a in the local function 
reference table 50a is '#000001 " is obtained. 

Below, a method of generating the above reference 
information in the reference generating unit 40 shown 
in Fig. 10 will be concretely explained. 
25 First, the space configuration generating the refer- 
ence information will be mentioned. 

Figure 19 is a view for explaining the space config- 
uration generating the reference information. 

As shown in Fig. 19. names such as "N1 and *N2* 
30 are given to the network nodes on their individual name 
spaces. Further, each network node is provided with in- 
terlaces of the Ethernet and ATM and each has a unique 
DNS name. 

Further, the in-domain names such as "P1 " and "P2" 
35 are given to the processes. 

Further, the in-domain names such as "FT, "F2". 
and "F3' are given to functions having the print names 
on the process "FT of "fund", 'func2", and "func3". and 
the in-domain names such as "F4" and "F5" are given 
•to to the functions having the print names on the process 
"P2" of B func4' and "funcS". 

Further, the processes "PT and *P2' and the func- 
tions "F1 • . 'F2" . "F3" , 'F4". and "F5" belong to the do- 
main "DT. Further, the functions "FT, "F2\ and 'F4* 
J 5 also belong to the domain "D2*. 

Here, the domains *D1 " and "D2" belong to the com- 
putation space "Cl 

In the example shown in Fig. 19, there are a man- 
agement process for executing the process "PT and a 
50 management process for generating the process "P2". 

The information concerning the space configuration 
is given to the reference generation units of these man- 
agement processes by the method of designating the 
information by arguments at the activation of the pro- 
55 gram of these management processes, implanting the 
information in the profram codes of these management 
processes, preparing a file in which the information is 
described and reading out from this file, etc. 



13 



25 



EP 0 794 493 A2 



26 



The reference generation unit of each management 
process initializes the domain reference table 53 (Si), 
initializes the local function reference table 50 (S2). in- 
itializes the process reference table 52 (S3), and initial- 
izes the remote function reference table 51 (S4) as 
shown in Fig. 20 based on this space configuration in- 
formation so as to generate the reference information. 
Below, a detailed explanation will be made of the steps 
shown in Fig. 20. 

[Initialization^ Domain Reference Table (Si)] 

The reference generation unit of the management 
process of the process "P1 ' generates the domain ref- 
erence table 53 based on information concerning the 
space configuration given in advance. 

Here, the reference generation unit generates the 
domain reference table 53 based on the following space 
configuration information of (1) and (2). 

(1 ) There is a domain "DT, but there is no super- 
domain of the same. 

(2) There is a domain "02", and the super-domain 
thereof is 'DT. 

First, the reference generation unit of the proc- 
ess "PT generates the domain reference table 53 
in which information concerning the domain "D1" is 
described as shown in Fig. 13 and. at the same 
time, generates the domain name hash table 54 in 
which the information concerning the domain "01" 
is described. 

Next,- the reference generation unit of the proc- 
ess "P1 ' adds the reference information concerning 
the domain "D2" to the domain reference table 53. 
At this time, the reference generation unit 40 ob- 
tains the reference information of the domain "01 
which is the higher domain of the domain "D2\ from 
the domain name hash table 54 shown in Fig. 13 
and adds the pointer to the domain "D2" to the do- 
main element reference list 91 of the domain "DT 
in the domain reference table 53 as shown in Fig. 
14 based on the reference information of this do- 
main "DT. 

[(Initialization of Local Function Reference Table (S2)] 

The reference generation unit of the management 
process of the process "P1 " prepares the local function 
reference table 50 shown in Fig. 1 5 for the process "P1 " 
based on the space configuration information of (3) to 
(5) shown below. 

(3) Afunction of a print name of "fund*, a local func- 
tion ID of "#000001", an in-domain name of "FT. 
and affiliated domain names of "D1 " and "D2". 

(4) A function of a print name of "func2", a local func- 



tion ID of "#000002". an in-domain name of "F2\ 
and affiliated domain names of *DT and "D2". 

(5) A function of a print name of "func3". a local func-. 
5 tion ID of "#000003", an in-domain name of "F3V 

and an affiliated domain name of "DT. 

Here, the local facility ID 62 in the local function 
reference table 50 shown in Fig. 23 is uniquely giv- 
en in each process. 
*o Further, the already prepared domain name 

hash table 54 shown in Fig. 21 is referred to using 
the domain names to which the functions belong to 
obtain the corresponding domain reference infor- 
mation. As shown in Fig. 23. the pointer to the ref- 
erence information of the function is added to the 
domain element reference list 91 of the affiliated do- 
main. 

Simultaneously with this, the reference gener- 
ation unit of the process *PT generates the local 
20 function name hash table 55 and the remote func- 
tion ID hash table 56 by adding the information of 
the functions as shown in Figs. 24A and 24B. 

Further, the reference generation unit of the 
process "P2" prepares the local function reference 
25 table 50 shown in Fig. 25 for the process "P2" based 
on the space configuration information of (6) and (7) 
shown below and, at the same time, generates the 
local function name hash table 55 and remote func- 
tion ID hash table 56 shown in Figs. 26A and 26B. 

30 

(6) A function of a print name of "func4", a local func- 
tion ID of "#000001". an in-domain- name of "F4". 
and affiliated domain names of "DT and "D2". 

3$ (7) A function of a print name of "func 1 ", a local func- 
tion ID of "#000002". an in-domain name of "F5". 
and an affiliated domain name of "Dr. 

[Initialization of Process Reference Table (S3)] 

jo 

The reference generation unit of the management 
process of the process "PT prepares the process ref- 
erence table 52 and the domain reference table 53 
shown in Fig. 27 and the process name hash table 55 
-5 shown in Fig. 29 for the processes "P1 " and "P2" based 
on the space configuration information of (3) to (11) 
shown below. 

(3) A process of a network node information of "N1 ", 
so a port number of '#10000'. an affiliated domain 
name of "DT, and a name inside the domain of 
•PT 

(9) A process of a network node information of "N2" ( 
« a port number of "#10001 \ an affiliated domain 
name of "DT, and a name inside the domain of 
"P2\ 
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(10) In 'NT, when the "machine name" is "ai.dvl. 
co.jp" a "communication media" is an Ethernet and 
when the "machine name" is "a2.dvl.co.jp" a "com- 
mmunication media" is an ATM. 

(11) In "N2", when the "machine name" is "bl.dvl. 
co.jp" a "communication media" is an Ethernet and 
when the "machine name" is "b1 dvl.co.jp" a com- 
munication media" is ATM. 

[Remote Function Reference Table (S4)] 

The reference generation unit of the management 
process of the process *P2" prepares the remote func- 
tion reference table 51 and domain reference table 53 
shown in Fig. 29 and the remote function name hash 
table 55 shown in Fig. 30 for the process "P2" based on 
the space configuration information of (12) and (13) 
shown below. 

(12) A print name of "func4", a process name of 
"P2", affiliated domains of "D1" and "D2", and a 
name inside the domain of "F4V 

(13) A print name of "fund", a process name of 
"P2", an affiliated domain of "D1", and a name inside 
the domain of # F5". 

Further, the reference generation unit of the 
management process of the process "P1 " prepares 
the remote function reference table 51 and domain 
reference table 53 shown in Fig. 31 and the remote 
function name hash table 55 show in Fig. 32 for the 
process "P1" based on the space configuration in- 
formation of (14) to (16) shown below. 

(14) A print name of "fund", a process name of 
■PT. affiliated domains of "D1" and "D2\ and a 
name inside the domain of "Fl V 

(15) A print name of "func2\ a process name of 
*Pl" : affiliated domains of "DT and "02", and a 
name inside the domain of "F2". 

(16) A print name of "func3", a process name of 
"P1 an affiliated domain of "D1 \ and a name inside 
the domain of "F3\ 

The remote function ID is obtained using the proto- 
col that the remote function ID is obtained when the 
■function name (set of domain name and name inside 
the domain)" is given. 

For example, the management process of the proc- 
ess "Pi" inquires about a function having a domain 
name of "Dl" and in-domain name of "F4" to the man- 
agement process of the process "P2\ The management 
process of the process "P2" obtains the reference infor- 
mation of the desired function by using the domain name 
hash table 54 and the function name hash table 55. 



Then, the management process of the process "P2" re- 
turns "#000100". which is the remote function ID of the 
function, to the management process of the process 
"P1\ 

s Next, an explanation will be made of the call oper- 
ation of a function using the reference information in the 
parallel distributed processing system by using Fig. 19 
as an example. 

Here, an explanation will be made by exemplifying 

io a case where the function "F4' of the domain name "D2" 
is called from the process "P1 " shown in Fig. 11 . 

First, the management process of the process "Pi " 
refers to the remote function name hash table 55 shown 
in Fig. 30 by using the in*domain function name "F4" as 

is the in-domain name 110 and obtains the domain refer- 
ence list 74 of the remote function reference table 51 
shown in Fig. 29 based on the function reference RFl. 
Then, the domain reference list 74 is examined and the 
item having the domain name of "D2" is found. At this 

20 time, it is seen from the process reference 72 of the re- 
mote function reference table 51 that the in-domain 
function name *F4" exists in the process "P2". 

Next, the management process of the process "P1 " 
designates a communication means for accessing an 

25 external process based on the process reference table 
52 shown in Fig. 27. In this case, by designating the 
Ethernet, the DNS host name of the process "P2" in 
which the process function "F4" exists is specified. 
Then, the management process of the process 'P1 ' ac- 

30 cesses the management process of the process "P2" by 
using this specified DNS host name and the network 
port ID 82 and performs the ID designation call by 
'#000100". which is the remote function ID 73 obtained 
from the remote function reference tabJe 51 shown in 

35 Fig. 29, based on the function reference RF 1 mentioned 
before. 

The management process of the process "P2" re- 
ceiving this call refers to the remote (unction ID hash 
table 56 on the process "P2" shown in Figs. 25A and 

40 26B by using "#0001 00", which is the designated remote 
facility ID, as a key and obtains the function reference 
LFl. Next, this management process obtains the desired 
reference information such as the local facility ID 62 for 
the function "F4" from the local function reference table 

•15 50 shown in Fig. 25 by using the function reference LF 1 . 
At this time, the local facility ID for the function # F4" is 
"#000001". The management process calls and exe- 
cutes the function "F4" based on this local facility ID and 
returns the result of execution to the management proc- 

50 ess of the process "P1 ". 

As explained above, according to the parallel dis- 
tributed processing system 31. by introducing the con- 
cept of computation space, the user can freely form 
management spaces for managing only the required 

55 functions in accordance with a particular purpose. As a 
result, the load on the user for management of the 
names and identifiers of the functions is reduced. Fur- 
ther, the number of the managed objects is decreased, 
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therefore the time for specifying an object can be short- 
ened. Further, erroneous access to the objects out of 
the reference space can be effectively prevented. 

Further, according to the parallel distributed 
processing system 31, by introducing domains in addi- 
tion to the computation space, the collision of names of 
the functions etc. is allowed even between domains and 
the degree of freedom of the functions useable in each 
domain is raised. 

Further, according to the parallel distributed 
processing system 31 . it is possible to flexibly and in 
addition easily deal with cases where a new function is 
added, a function is deleted, etc. by amending the por- 
tions concerning the functions, for example, the local 
function reference table 50. remote function reference 
table 51, process reference table 52, domain reference 
table 53, domain name hash table 54, function/process 
name hash table 55. and remote function ID hash table 
56. 

Further, in the above parallel distributed processing 
system 31 , the type of the execution environment (OS) 
of the network nodes N1 and N2 shown in Fig. 19 is ir- 
relevant to the space management, therefore a system 
not dependent upon the execution environment can be 
provided. 

Further, in the above parallel distributed processing 
system 31 . by using the concept of domains, it is possi- 
ble to manage the references of the functions provided 
in each process without regard as to the mode of con- 
nection of the network nodes in the network. 

The present invention is not limited to the above em- 
bodiments. For example, the types of the computation 
objects comprising ihe domains are not limited to those 
mentioned above. 

Further, in the above embodiments, functions were 
exemplified as the computation objects, but other than 
this, the present invention can be similarly applied even 
if reference information such as computation modules 
for executing predetermined programs, instances, 
classes, methods, global shared functions, and filed are 
managed. 

Further, it is also possible to provide the reference 
holding unit and the reference generation unit for every 
process or provide them for every computation object. 

Further, in the above embodiments, as shown in 
Fig. 16. Fig. 17, Fig. 21, Fig. 22. Fig. 24A. Fig. 24B. Fig. 
26A. Fig. 26B. Fig. 30, and Fig. 32, the in-domain name 
indicating the name of the object was used as the refer- 
ence information, but it is also possible to use an iden- 
tifier indicating the location of the computation object as 
the reference information. 

Third Embodiment 

[Space Management System] 

First, the space management system adopted in a 
parallel distributed processing system according to the 



present embodiment is the same as the space manage- 
ment system in the parallel processing system accord- 
ing to the second embodiment shown in Fig. 9 explained 
above. The concepts of the "reference space", "refer-. 
s ence", and 'domain' are the same as well. 

Note that the domains are not shown in Fig. 9, but 
are shown in Fig. 36. 

Figure 33 is a view of the configuration of a parallel 
distributed processing system 31 according to the 
io present embodiment. 

As shown in Fig. 33, in the parallel distributed 
processing system 31 . network nodes 33 to 35 are con- 
nected via a network 32. 

In the network node 33, a process (partial space) 
is 36 executed by the management process 38 exists. 

The management process 38 has a reference hold- 
ing unit 39, a reference generation unit 40, and a proc- 
ess allocation unit 42. 

The reference holding unit 39 stores reference in- 
formation identifying the processes, computation ob- 
jects, and domains for specifying their locations and 
controlling and managing the same. 

The reference generation unit 40 generates the ref- 
erence information mentioned later from the reference 
information on the program modules obtained from an' 
export module and an import module and the given al- 
location information. Here, a "program module" means 
a collection of programs for executing a process and can 
also be comprised by a plurality of files. For example, it 
is possible even if the program module "X" is comprised 
by the two files of "ps.exe" and "px.d11". 

Each program module encompasses an export 
module indicating the content of registration to an inter- 
nal reference table and an import module indicating the 
content of registration to an external reference table for 
giving the reference relationship on the program mod- 
ules to the reference generation unit 40. In the example 
shown in Fig. 34. in the function func2() of the program 
module "X". the function func4() of the program module 
"Y" is read. Further, the function fund () is read from the 
function fund () of the program module "Y". At this time, 
"export (fund):" is described in the export module 202 
of the program module "X", and "import (Y; func4) ;" is 
described in the import module 203. As shown in Fig. 
34, the programmer describes "call (Y: func4, args)" 
where a catl of the external function is described. This 
means that the function func4 of the program module 
"Y" is called by the arguments indicated by args. In this 
way. the description of the program concerning the ref- 
erence on the program is described unrelated to the al- 
location at the time of execution of the program. 

Note that, the export module and import module are 
automatically generated from the application source 
code or generated under at the responsibility of the ap- 
plication programmer by using a translator or the like. 

The process allocation unit 42 executes the pro- 
gram module on the designated network node based on - 
the given allocation information to generate a process. 
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The process allocation unit 42 is provided with a 
process allocation table 210 and a port number assign- 
ment table 211 as shown in Fig. 35. 

. (Allocation Information] s 

The allocation information is comprised by informa- 
tion for determining in which network node which pro- 
gram module is allocated and information concerning 
the names given to the processes and the computation io 
objects. — • . 

- This allocation information is input to a master man- 
agement process by for example a reading of a file, an 
input operation of the user, or the like. Information in the 
allocation information necessary for preparing the proc- 
ess allocation table and the port number assignment ta- 
ble is output from the master management process to a 
slave management process. 

Here, for convenience of explanation, the manage- 
ment process reading the allocation information first will 20 
be referred to as a master management process. As op- 
posed to this, a management process receiving the 
process generation command from the master manage- 
ment process will be referred to as a slave management 
process. 2s 

Below, an explanation will be made of a specific ex- 
ample of the allocation information by taking as an ex- 
ample a case where the computation space and process 
allocation respectively have the structures shown in Fig. 
36 and Fig. 37. 30 

As shown in Fig. 36. the processes "PX1 * and "PZ1 " 
are allocated in the network node "A', a process "PY1 " 
is allocated In a network node "B*. and a process "PZ2" 
is allocated in a network node "C". 

Here, a computation space "Cl" is composed by the 35 
processes *PXT. "PZT , "PVT. and "PZ2" 

Further, a domain "DT is comprised by processes 
•PXV and "PZ1". and a domain "D2" in comprised by 
processes 'PY1 " and "PZ2". 

Here, the process "PXT is generated by executing -»o 
the program module 'X" shown in Fig. 34, the process 
•PYT is generated by executing the program module 
"Y\ and the processes 'PZl " and "PZ2" are generated 
by executing the program module "Z\ 

Further, as shown in Fig. 37, the network node "A" J 5 
is connected to an Ethernet 220 and an ATM 221 and 
has the network addresses {DNS name) •s1.dvl.co.jp" 
and "s1a.dvl.co.jp" , respectively. 

The network node "B" is connected to the Ethernet 
220 and ATM 221 and has 'theVietwork addresses so 
's2.dvl.co.jp' and 's2a.dvl .co.jp', respectively. 

The network node "C" is connected to the Ethernet 
220 and has the network address *a3.dvi .co.jp". 

Further, as shown in Fig. 37, in the network nodes 
"A", *B" ( and "C", management processes A, B, and C ss 
respectively exist. 

In the above cases shown in Fig. 36 and Fig. 37. 
the allocation information has the following information. 



• The program module "X" is allocated on the network 
node "A" and is named the process "PX1 

• The program module *Y* is allocated on the network 
node *B* and is named the process "PYl V 

• The program module *Z" is allocated on the network 
node "A' and is named the process *PZ1 ". 

• The program module *Z* is allocated on the network 
node "C" and is named the process "PZ2". 

• The program module "X" location (real) is: 

"ftp://ftp.dvl.co.jp/bin/px.exe". 

• The program module 'Y" location is: 

'ftp://ftp.dvl.co.jp/bin/py.exe* and 
"ftp://ftp.dvl co.jp/bin/py.d1 1 ". 

• The program module 'X" location is: 

"ftp://ftp.dvl.co.jp/bin/p2.exe". 

• In the network node "A", the network address is 

*s1 dv1 .co.jp* for the Ethernet (media) and 
"sla.dvl .co.jp* for the ATM (media). 

• In the network node *B*. the network address is 
"s2.dvl.co.jp* for the Ethernet (media) and *s2a. 
dv1 .co.jp* for the ATM (media). 

• In the network node "C", the network address is 
"s3.dvl.co.jp" for the Ethernet (media). 

[Management Process] 

The management process is activated in advance 
when allocating the processes in the parallel distributed 
processing system in a distributed manner. 

The management process is a type of process in 
which a fixed port of each network made (for example 
a port number "010000") is assigned and which per- 
forms the allocation of general process based on the al- 
location information. 

The general processes and the management proc- 
ess are the same in terms of facility, but different in the 
point that a fixed port is assigned to the management 
process. 

In Fig. 37 : the allocation information is given to the 
management process C, and the process generation 
command is output from the management process C to 
the management processes A and B. 

Accordingly, the management process C becomes, 
the master management process, and the management 
processes A and B become the slave management 
processes. 

The processing of the master management process 
C and the slave management processes A and B are 
shown in for example Fig. 7. 

[Process Allocation Table] 

A process allocation table 210 is provided in a proc- 
ess allocation unit 42 and stores the allocation informa- 
tion or the required information among the allocation in- 
formation. 

Here, where there are a plurality of allocation infor- 
mation, in order to identify the allocation information, a 
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configuration ID is given to each allocation information. 

Figure 39 is a view for explaining the process allo- 
cation table 210. 

As shown in Fig. 39, the process allocation table 
210 is comprised by a configuration ID 231, a network 
address 232 of a master node, a node information 233 ; 
a program information 234, and a program allocation in- 
formation 235. 

The node information 233 is a table of a node name 
236, netmork address 237, and media 238. 

The program infomation 234 is a table of a program 
name 239 an3 a program location information 240. 

The program allocation information 235 is a table of 
a process name 241 , a program name 242, and a node 
name 243. 

In the parallel distributed processing system shown 
in Fig. 37, the process allocation units 42 of the network 
nodes A, B, and C are respectively provided with proc- 
ess allocation tables 210 as shown in Fig. 41, Fig. 42. 
and Fig. 40. The process allocation table 21 0 of the net- 
work node C is generated by the management process 
C based on the allocation information. The process al- 
location tables 210 of the network nodes A and B are 
generated by the management processes A and B 
based on the required information among the allocation 
information. 

[Port Number Assignment Table] 

Figure 43 is a view for explaining a port number as- 
signment table 211. 

As shown in Fig. 43, the port number assignment 
table 211 is comprised by a configuration ID 250. a proc- 
ess name 251 , and a port number 252. 

In the present embodiment, as shown in Fig. 44 in 
the slave management processes A and 3 and the mas- 
ter management process C. port number assignment ta- 
bles 253. 254. and 21 1 are respectively generated. The 
port number assignment tables 253 : 254. and 211 are 
provided in the process allocation units 42 of the net- 
work nodes A. B, and C, respectively. 

[Reference Infomation] 

This reference information is generated by an im- 
port processing and an export processing mentioned 
later. 

In the following example, a case is shown where 
functions allocated distributed throughout a network are 
referred to. The reference information is comprised of 
four reference tables and three search use hash tables 
in the second embodiment as shown in Fig. 11. As the 
reference tables, there are a local function reference ta- 
ble 50 shown in Fig. 12. a remote function reference ta- 
ble 51 shown in Fig. 13. a process reference table 52 
shown in Fig. 14, and a domain reference table 53 
shown in Fig. 15. Further, the search use hash tables 
are hash tables between the names and identifiers and 



include a domain name hash table 54 shown in Fig. 16. 
a function/process name hash table 55 shown in Fig. 
1 7, and a remote function ID hash table 56 shown in Fig. 
18. 

s The contents of these tables are the same as ex- 
plained in the second embodiment. 

Note that, as the network node 34. there is a proc- 
ess (partial space) 37 executed by the management 
process 41. This is conceptually the same as the net- 
to work node 33. 

(Processing in Management Process] 

Below, an explanation will be made of the process- 
's ings in the master management process C and the slave 
management processes A and B while referring to Fig. 
38. 

Figure 38 is a flowchart of the processings in the 
master management process C and the slave manage- 
20 ment processes A md B. 

First, when a plurality of allocation information are 
input to for example the master management process 
C. a configuration ID is given to each allocation informa- 
tion for the master management process C to identify 
25 the allocation information (step Si). 

Next, the master management process C generates 
the process allocation table 21 0 as shown in Fig. 49 from 
the allocation information (step S2). 

The master management process C outputs the in* 
30 formation for generating the process in each network 
node among the allocation information to the slave man- 
agement processes A and B (step S3). The slave man- 
agement processes A and B store the information input 
from the master management process C in the process 
3$ allocation tables (step S9). 

The master management process C outputs the 
process generation command as shown below to the 
slave management processes A and B itself (step S4). 

Namely, the master management process C out- 
40 puts a process generation command for instructing the 
generation of the processes having a configuration ID 
of #000001 and process names of "D1/PX1" and 
*D1/PZT to the slave management process A. 

Further, the master management process C outputs 
J5 a process generation command for instructing the gen- 
eration of the process having the configuration ID of 
#000001 and process name of "D2/PY1" to the slave 
management process B. 

Further, the master management process C outputs 
so an instruction for generating the process having the con - 
figuration ID of #000001 and process name of "D2/PZ2: 
to itself. At this time, the master management process 
C performs both functions of a master and a slave. 

The slave management porocesses A and B obtain 
55 the program for realizing the corresponding processes 
from the process allocation table when the process gen- 
eration command is input (step S4) and further obtain - 
the location information of the program from the pro- 
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gram name and start the download of that program 
(Step S10). As the protocol of the download, "ftp. http" 
or the like is used. 

Then, the slave management processes A and B 
uniquely assign a port number for every network node s 
at the process generation (step S11) and give the as- 
signed port number and configuration ID to the program 
module as the arguments and generate processes (step 
S 1 2). This assigned port number is registered in the port 
number assignment table 21 1 provided in each process io 
allocation unit (step S1 3). Also the master management 
process C receiving the process generation command 
from itself performs similar processing. 

The slave management processes A and B and the 
general process in the network node C output an ending 
message to the master management process C after the 
ending of the generation of the process. 

When the master management process C decides 
from these ending messages that all of the process gen- 
eration is ended (step S5), it outputs the import start 20 
command to the slave management processes A and B 
and the general processes in the network node C (step 
S6). 

At this time, the master management process C out- 
puts the import start command with respect to the proc- 2s 
esses 'D1/PX1 " and "D2/PZ1 ' of the configuration ID of 
#000001 to the slave management process A. 

Further, the master management process C outputs 
the import start command with respect to the process 
"D2/PY1 ' of the configuration ID of #000001 to the slave 30 
management process B. 

On the other hand, the master management proc- 
ess C outputs the import start command with respect to 
the process "D2/PZ2" of the configuration I D of #000001 
to the general process of the network node C 35 

The slave management processes A and B obtain 
the corresponding port numbers from the port number 
assignment table and output the import processing com- 
mand to the general processes (step S14). When re- 
ceiving the import processing command (step S23 -*o 
shown in Fig. 45), the general process starts the import 
processing (step S24), and when the import processing 
is ended, outputs the ending message to the master 
management process C. 

The master management process C decides wheth- 4 * 
er or not the import processing is all ended (step S7), 
and where it decides that all has been ended, outputs 
the process execution start command to the slave man- 
agement processes A and B and the general process of 
the network node C (step S8). so 

At this time, the master management process C out- 
puts the process (main function) execution start com- 
mand with respect to the processes "D1/PXV and 
"D1/PZ1 " of the configuration ID of #000001 to the slave 
management process A. ss 

Further, the master management process C outputs 
the process execution start command with respect to the 
process "D2/PY1 " of the configuration ID of #000001 to 



the slave management process B. 

On the other hand, the master management proc- 
ess C outputs the process exexution start command 
with respect to the process "D2/PZ2" of the configura- 
tion ID of #000001 to the general process of the network 
node C. 

The slave management processes A and B output 
the process execution start command to the corre- 
sponding general processes (step S15). Namely, the 
slave management processes A and B obtain the cor- 
responding port number from the port number assign- 
ment table and output the call command of the main 
function. 

Figure 45 is a flowchart of the processing of a gen- 
eral process. 

As shown in Fig. 45. the general process is activat- 
ed by receiving as its input the port number, configura- 
tion ID, and network address of the master node as the 
arguments from the management process of the same 
network node (step S21). Then, next, the general proc- 
ess executes the export processing (step S22). 

Then, it is decided whether or not there is an import 
processing command from the master management 
process C and the slave management processes A and 
B (step S23), and where it is decided that it exists, the 
import processing is executed (step S24), and the proc- 
ess execution start command is awaited (step S25). 
When the process execution start command is inputted, 
the general process executes the main function (step 
S26). 

[Export Processing] 

Below, the explanation will be made taking the ex- 
port processing concerning the process "PX1 " shown in 
Fig. 37 as an example. 

First, the general process calls "export (fund)" 
shown in Fig. 34, that is. the export module on the proc- 
ess "PX1 " shown in Fig. 37, and registers "fund ' in the 
print name 61 of the local function reference table 50 as 
shown in Fig. 51 . 

In the local function reference table 50, as the local 
function ID 62 of "func 1 ". the configuration ID "#000001 " 
shown in the port number assignment table 253 of Fig. 
44 is registered, and "#00100" is registered as the re- 
mote function ID 63. This remote function ID "#00100" 
is registered in also the remote function ID hash table 
56 shown in Fig. 52B. 

Further, a name is not attached to the function 
fund" on the name space, therefore the in-domain 
name 64 of the local function reference table 50 be- 
comes "nil". Further, this function "fund" does not be- 
long to any domain, and the domain reference list 65 
becomes 'nil". 

The local function reference table 50 of the process 
"PY1" shown in Fig. 37 becomes as shown in Fig. 53. 

Further, the local function reference table 50 of the 
processes "PZ1 and PZ2 shown in Fig. 37 becomes as 
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shown in Fig. 54. 

Note that, in the export processing, also the func- 
tion/process name hash table 65 shown in Fig. 52A is 
registered. 

Here, as mentioned above, the import processing 
•and export processing for the process 'PXT shown in 
Fig. 37 were exemplified, but also the import processing 
and export processing of the 'PZ1', "PYT, and "PZ2" 
shown in Fig. 37 are carried out by similar procedures. 

[Import Processing] 

Figure 46 is a flowchart for explaining the import 
processing. 

The general processing of the network nodes A. B, 
and C first perform the communication with the master 
management process when the import processing is 
started, call the import module contained in the program 
module, and obtain the process name of the corre- 
sponding process and the allocation destination node 
name thereof from the program module name based on 
the process allocation table 21 0 (step S31 ). At this time, 
where there are a plurality of corresponding processes, 
all are regarded as the objects of the import processing. 

For example, in the case of the process 'PXT 
shown in Fig. 31, 'import (Y: func4)* contained in the 
import module of the program module "X* is called. 
Then, the corresponding process name "D2/PY1" and 
node name "B" are obtained from the program allocation 
information 235 of the process allocation table 210 
shown in Fig. 40 by using the program module name "Y" 
contained in this "import (Y: func4)" as the key. 

Next, this general process decides whether or not 
the obtained process name has been already registered 
in the process reference table 52 (step S32), and where 
it has not been registered, further decides whether the 
node name has been registered in the process refer- 
ence table 52 (step S35). Where the node name has not 
been registered, the general process performs the com- 
munication with the master management process, ob- 
tains the information concerning the corresponding 
node, and registers this node information in the process 
reference table 52 (step S33). 

For example, as shown in Fig. 47, the process name 
"D2/PY1 ' has not been registered in the process refer- 
ence table 52, therefore this process is registered in the 
process reference table 52. At this time, so as to specify 
the network address receiving the import, the node in- 
formation is registered in the process reference table 52 
by using the process allocation table 210 shown in Fig. 
40. From this node information, the network address re- 
ceiving the import in viewed. 

More specifically, from the process allocation table 
210 shown in Fig. 40, the node information that, in the 
network address of the network node "B\ Ethernet is 
's2.dvl.co.jp' and ATM is "s2a.dv1.co.jp', is obtained. 

Next, the general process makes an inquiry to the 
management process on the network node receiving the 



import by using the configuration ID and process name 
as a key and obtains the corresponding port number 
(step S36). Concretely, the general process asks the 
port number to the management process receiving the 

5 import by using the configuration ID = #000001 and the 
process name '02/PYt* aa a key. The management 
process receiving the inquiry returns the port number 
'#01 0000* to the general process by referring to the port 
number assignment table 254 shown in Fig. 44. 

10 When obtaining the port number, the general proc- 
ess registers this port number and process name in the 
process reference table 52 (step S37). By this, the proc- 
ess reference table 52 becomes as shown in Fig. 43. 
Further, where the reference of the domain "D2" to 

is which the process "PYT belongs does not exist, as 
shown in Fig. 48, a new domain is prepared in the do- 
main reference table 53. 

Next, as shown in Figs. 49A and 49B, a function/ 
process name hash table 55 and a domain name hash 

20 table 54 are prepared, and a reference relationship is 
established. 

Next, the general process obtains the reference of 
the corresponding process from the process name and 
obtains a remote ID by using the print name as a key 

2S (step S33). 

Concretely, it makes an inquiry to the remote ID, for 
example : the management process, by using the print 
name "func4" of the function described in the import 
module as a key The management process obtains the 

30 remote ID '#0001 0T from the local function reference 
table by using the print name "func4' as a key and re- 
turns this to the genera! process. Note. that, the local 
function reference table has been prepared in the export 
processing (step S22) performed before the import 

3S processing (step S24) in started as shown in Fig. 45. 

Then T the general process registers the obtained re- 
mote ID. print name, and process reference in the re- 
mote function reference table (step S34). 

Concretely, the general process registers the ob- 

-io tained remote ID "#000 1 01". print name "func4\ and the 
process reference in the remote function reference table 
51 as shown in Fig. 50 (step S34). 

[Execution of Program Module] 

•is 

For example, when the network node A shown in 
Fig. 37 executes the program module X shown in Fig. 
34 and generates the process "PXT. the processing 
corresponding to the "call (Y; func4)" in the function 

so "func2{)' of the program module X is executed. At this 
time, the process "PX1" obtains the process reference 
concerning the process of the program module Y by re- 
ferring to the process allocation table 210 shown in Fig. 
41 of the network node A. This process reference indi- 

ss cates that the program module Y is allocated in the net- 
work node B and the network address of the network 
node 3 is 's2.dvl.co.jp' for the Ethernet and "s2a. - 
dvl.co.jp' for ATM. 
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Next, the process "PXV obtains the external refer- 
ence by referring to the remote function reference table 
51 shown in Fig. 50 by using a set of the obtained proc- 
ess reference and the print name f unc4" and the com- 
. munication media as a key. This external reference is s 
the remote function ID "#000101 the network address 
and the network port ID '#010000". 

Next, the process "PX1 ' performs the access to the 
other process by using the network address and net- 
work port ID and performs the function call by the des- 
ignation of the-femote function ID. 

- The called process obtains the local ID from the re- 
mote function ID hash table 56 shown in Fig. 52B by 
using the remote ID as a key and out the function acti- 
vation command to the system (OS). 

As explained above, in the parallel distributed 
processing system 31 , the allocation information con- 
cerning the network node by which the program module 
is executed is described outside of the program module, 
therefore the network node for which the program mod- 
ule is executed can be dynamically changed by only 
changing the allocation information without amendment 
of the description of the program module. Namely, after 
the computation object space is formed, the space con- 
figuration thereof can be dynamically changed. 

Further, according to the parallel distributed 
processing system 31, the application programmer can 
describe the reference relationship at the time of exe- 
cution of the program module without depending upon 
the network allocation at the time of execution. 

Namely, the allocation information at the time of ex- 
ecution of the program module is described outside of 
the program module, therefore it is sufficient so far as 
the user describes the program module by being con- 
scious of only the reference relationship of the compu- 
tation object on the program module (set of the program 
name and computation object). For this reason, a static 
reference relationship of the computation object de- 
scribed in the program module and the dynamic refer- 
ence relationship described in the allocation information 
can be separately handled. 

Further, as explained above, according to the par- 
allel distributed processing system 31 , by introducing 
the concept of computation space, the user can freely 
form management spaces for managing only the re- 
quired functions in accordance with a particular pur- 
pose. As a result, the load on the user for management 
of the names and identifiers of the functions is reduced. 
Further, the number of the managed objects is de- 
creased, therefore the time for specifying an object can 
be shortened. Further, erroneous access to the objects 
out of the reference space can be effectively prevented. 

Further, according to the parallel distributed 
processing system 31, by introducing domains in addi- 
tion to the computation space, the collision of names of 
the functions etc. is allowed even between domains and 
the degree of freedom of the functions useable in each 
domain in raised. 



Further, according to the parallel distributed 
processing system 31, it is possible to flexibly and in 
addition easily deal with cases where a new function is 
added, a function is deleted, etc. by amending the por- 
tions concerning the functions, for example, the local 
function reference table 50. remote function reference 
table 51, process reference table 52, domain reference 
table 53. domain name hash table 54, function/process 
name hash table 55, and remote function ID hash table 
56. 

Further, in the above parallel distributed processing 
system 31 . the#type of the execution environment (OS) 
of the network nodes is irrelevant to the space manage- 
ment, therefore a system not dependent upon the exe- 
cution environment can be provided. 

Further, in the above parallel distributed processing 
system 31 . by using the concept of domains, it is possi- 
ble to manage the references of the functions provided 
in each process without regard as to the mode of con- 
nection of the network nodes in the network. 

The present invention is not limited to the above em- 
bodiments. For example, the types of the computation 
objects comprising the domains are not limited to those 
mentioned above. 

Further, in the above embodiments, function were 
exemplified as the computation objects, but other than 
this, the present invention can be similarly applied even 
if reference information such as functions, instances, 
classes, methods, global variables, and files for execut- 
ing predetermined programs are managed. 

Further, any management process among the man- 
agement processes shown in Fig. 37 can become the 
master management process. 

Further, it is also possible to provide the reference 
holding unit and the reference generation unit for every 
process or provide them for every computation object. 

Fourth Embodiment 

As this embodiment, there is shown a reference 
conversion device for converting the reference informa- 
tion of processing modules designated by the applica- 
tion level to system reference information capable of di- 
rectly specifying an object on an actual system so as to 
perform for example communication among the 
processing modules in an environment where for exam- 
ple computation spaces comprised by a plurality of 
processing modules are allocated on a computer net- 
work ccmprised of a plurality of computer devices linked 
with each other and where each processing module per- 
forms distributed processing or parallel distributed 
processing while substantially communicating with an- 
other module. 

Figure 55 is a block diagram of the configuration of 
the reference conversion device. 

The reference conversion device 530 has an input 
switching unit 531 , a first conversion unit 532, a second - 
conversion unit 533, a third conversion unit 534, a fourth 
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conversion unit 535, a fifth conversion unit 536, an out- 
put switching unit 537. and a control unit 538. 

Note that, in Fig. 55, a symbol A indicates applica- 
tion reference information, and a symbol S indicates 
system reference information. 

First, an explanation will be made of the configura- 
tion and functions of the units. 

The input switching unit 531 selectively inputs the 
input application reference information or the applica- 
tion reference information input again from the output 
switching unit_537 mentioned later to one of the first con- 
version unit 532 to the fifth conversion- unit 536 based 
on a switching signal from the control unit 538. 

The first conversion unit 532 converts the input ap- 
plication reference information to one piece of system 
reference information. 

The second conversion unit 533 converts the input 
application reference information to a set of two or more 
pieces of system reference information. 

The third conversion unit 534 converts the input ap- 
plication reference information to other application ref- 
erence information. 

The fourth conversion unit 535 converts the input 
application reference information to a set of two or more 
pieces of application reference information. 

The fifth conversion unit 536 converts the input ap- 
plication reference information to a set of the other ap- 
plication reference information and system reference in- 
formation. 

The output switching unit 537 switches among the 
processing of the reference information which is con- 
verted at the first conversion unit 532 to the fifth conver- 
sion unit 536 and sequentially output based on the con- 
trol signal input from the control unit 538. Namely, it per- 
form the switching so that the system reference infor- 
mation among the reference information to be output is 
output to the outside and so that the application refer- 
ence information is input to the input switching unit 531 
again. Note that, the output switching unit 537 has a not 
illustrated storage unit therein and that the system ref- 
erence information which is converted and can be out- 
put to the outside is stored once in this storage unit. 
Then, at the point of time when all of the reference in- 
formation with respect to the application reference infor- 
mation input to the reference conversion device 530 are 
converted to the system reference information, the sys- 
tem reference information are output together contain- 
ing also the information stored in the storage unit. 

The control unit 538 controls the input switching unit 
531 and the output switching unit 537 so as to suitably 
convert the input application reference information to 
the system reference information. 

The control unit 538 first reads the application ref- 
erence information input to the input switching unit 531 
from the outside, that is. the application reference infor- 
mation to be converted, or the application reference in- 
formation input from the output switching unit 537. ex- 
amines the contents thereof, and determines in which 



conversion unit among the first conversion unit 532 to 
the fifth conversion unit 536 the reference solution would 
be suitably carried out. Then, based on the result of the 
decision, it switches the input switching unit 531 so that 

s the input application reference information is suitably in- 
put to a desired conversion unit. 

Further, the control unit 538 decided whether the 
reference information resulting from the conversion in- 
put from the first conversion unit 532 to the fifth conver- 
ge sion unit 536 to the output switching unit 537 is the ap- 
plication reference information or the system reference 
information and, if it is the application reference infor- 
mation, controls the output switching unit 537 so that 
this information in input to the input switching unit 531 

15 again. Further, if it is the system reference information, 
it controls the output switching unit 537 so as to store 
the same in the storage unit in the output switching unit 
537 as the reference information for output. 

Further, the control unit 538 outputs the system ref- 

20 erence information or the set of the system reference 
information as the result of conversion to the outside if 
the reference information with respect to the input ap- 
plication reference information are all converted to the 
system reference information. 

25 An explanation will be made next of the operation 
for substantially controlling the reference conversion de- 
vice 530 by the control unit 538 by referring to the flow- 
chart of Fig. 56. 

As shown in Fig. 56. the control unit 538 starts the 

30 processing when the application reference information 
is input to the reference conversion device 530 (step 
S110), examines in which of the first conversion unit 532 
to the fifth conversion unit 536 the conversion of the in- 
put application reference information is possible, and in- 

35 puts the application reference information to the conver- . 
sion unit the making the conversion unit start the con- 
version (step S1 1 1 ). Then, when the conversion is end- 
ed, it checks whether or not the application reference 
information exists in the reference information resulting 

■to from the conversion (step Si 1 2) and, where it exists, 
repeats the processing of step S 1 1 1 again for the appli- 
cation reference information. 

When it is decided at step Si 12 that all reference 
information are converted to the system reference infor- 

^5 mation, the result of the conversion is output (step S1 53) 
and the series of conversion processing is ended (step 
S114). 

Next, an explanation will be made of the operation 
of this reference conversion device 530 by referring to 
50 Figs. 57 A. 57B. and 57C 

First, an example of the usual operation of this ref- 
erence conversion device 530 will be shown in Fig. 57A. 

In the example shown in Fig. 57 A. first the input ap- 
plication reference information A is examined in content 
55 in the control unit 538, determined to be converted at 
the founh conversion unit 535, and converted to the set 
of the application reference information (Al. A2. A3) at 
the fourth conversion unit 535. The result of this conver- 
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sion is all application reference information, therefore 
the conversion will be carried out again, the content of 
the application reference information to checked again 
by the control unit 538. and the next conversion unit is 
' determined. In the example of Fig. 57 A, the application s 
reference information A1 to A3 are separately input to 
the first to third conversion units 532 to 534 where the 
conversion in carried out. 

As the result of conversion, the application refer- 
ence information A1 is converted to the system refer- 10 
ence information' Si at the first conversion unit 532, the 
application reference information A2 is converted to the 
set of the system reference information (S2, S3) at the 
second conversion unit 533, and the application refer- 
ence information A3 is converted to the application ref- '5 
erence information A4 at the third conversion unit 534. 

Among these results of conversion, only the refer- 
ence information A4 is still application reference infor- 
mation, therefore the conversion is carried out again. 
Namely, the selection of the conversion unit is carried 20 
one again based on the contents thereof and, as a re- 
sult, at this time, it is converted to the set of the applica- 
tion reference information and system reference infor- 
mation (A5. S4. S5) at the fifth conversion unit 536. Fur- 
ther, the application reference information A5 remains 25 
in this conversion result, therefore this is now converted 
to the system reference information S6 in the first con- 
version unit 532. 

The conversion is sequentially carried out in this 
way. As a result, the input application reference infor- 30 
mation A is converted to a series of the system reference 
information (Si, (S2.S3), (S6. S4, S5)) and output. 

Note that, the application reference information and 
the system reference information are specifically the 
logical names or IDs of the modules and objects. 

Further, in the first conversion unit 532 to the fourth 
conversion unit 535. it is also possible to perform the 
processing together for a set of the reference informa- 
tion. An example of this is-shown in Fig. 57B. 

In Fig. 57B, a set of the application reference infor- 
mation (A1. A2, A3) is generated from the input appli- 
cation reference information A by the fourth conversion 
unit 535. Next, this set of application reference informa- 
tion is input to the first conversion unit 532 as it is and 
a corresponding set of system reference information 
(Si, S2. S3) is obtained. 

I n the first conversion unit 532 to the fifth conversion 
unit 536 of the reference conversion device 530, the set 
of the application reference information can therefore be 
determined as the input in this way. 

Further, it is also possible to input only one part of 
the set of the application reference information without 
inputting the set of the application reference information 
together as shown in Fig. 57B. 

Such an example is shown in Fig. 57C. 

Also in Fig. 57C. a set of the application reference 
information (A1 , A2, A3) in generated from the input ap- 
plication reference information A by the fourth conver- 



sion unit 535. Then, in the next stage, the set of the ap- 
plication reference information (At. A2) is extracted, 
and the application reference information A3 remaining 
in the first conversion unit 532 is output to the second 
conversion unit 533 and converted again. 

In the reference conversion device 530, it is also 
possible to appropriately rearrange the sets of the ap- 
plication reference information generated in this way 
and define the same as the input of the application ref- 
erence information of next stage. 

In this way, in the reference conversion device 530. 
the first conversion unit 532 to the fifth conversion unit 
536 are suitably repeatedly selected and the conversion 
of the reference information is performed based on the 
contents of the reference information for the input appli- 
cation reference information. Accordingly, even applica- 
tion reference information having a complex structure is 
developed as the series of the system reference infor- 
mation finally directly indicating the object on the sys- 
tem. 

Fifth Embodiment 

The parallel distributed processing system of this 
embodiment applies the reference conversion device 
explained as the fourth embodiment to the parallel dis- 
tributed processing system shown in Fig. 9 as the sec- 
ond embodiment. The processing modules perform the 
processing by substantially linking up. 

In the parallel distributed processing system shown 
in Fig. 9. the parallel objects communicate while regard- 
ing the message transmission as a base-and so parallel 
distributed processing is realized. . 

In the parallel distributed processing system of this 
embodiment, in this message transmission, the node re- 
ceiving the transmission, process, and object are spec- 
ified from the remote reference with respect to the object 
receiving the transmission, and the processing of trans- 
mission is carried out. At this time, if the destination of 
transmission has become a domain or object cluster, a 
multicast is carried out. 

Further, if the reference unit indicates a proxy ob- 
ject, the proxy is relayed to carry out the message trans- 
mission. Further according to some locations of the 
destination of transmission, the RPC on the network, 
IPC inside the node, or in-process communication is se- 
lected to carry out the message transmission: 

At this time, the parallel object manages the mutual 
remote reference in the application space and high 
speed message transmission is realized by this. An ex- 
planation will be made below of the object reference in- 
formation in the application space in such a distributed 
processing system and the reference conversion device 
for performing the conversion with the system reference 
information for performing the actual transmission. 

Figure 58 is a block diagram of the configuration of 
the reference converter 150. 

The reference converter 650 has a conversion con- 
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trol unit 651. a domain conversion unit 652, an entity 
conversion unit 653, a process conversion unit 654, a 
node name conversion unit 655, and a DNS name proxy 
conversion unit 656. 

First, an explanation will be made of the reference s 
information referred to from the units of the domain con- 
version unit 652 to the node name conversion unit 655 
and for the space management of the distribute 
processing system embodying the invention. 

The main reference information used in this distrib- io 
uted processing system is managed by four reference 
. tables and three search use hash tables as shown in 
Fig. 11. As the four reference tables, there are a local 
function reference table 50 shown in Fig. 12, a remote 
function reference table 51 shown in Fig. 13 ; a process '5 
reference table 52 shown in Fig. 14, and a domain ref- 
erence table 53 shown in Fig. 15. Further, the search 
use hash table is a hash table between the names and 
identifiers and is a domain name hash table 54 shown 
in Fig. 1 6, a function/process name hash table 55 shown 20 
in Fig. 1 7, and a remote function ID hash table 56 shown 
in Fig. 18. 

The contents of these tables are the some as ex- 
plained in the second embodiment above. 

Next, an explanation will be made of the configura- 25 
tion of the units of the reference convener 650 shown 

in Fig. 53. 

The conversion control unit 651 performs the over- 
all management of the reference convener 550. More 
specifically, it confirms the contents of the input applica- 30 
tion reference information and outputs the input name 
to one of the domain conversion unit 652 to the node 
name conversion unit 655 in accordance with the type 
thereof. Then, it reads the conversion result obtained at 
the conversion unit and checks whether or not the ob- 35 
tained reference information is the system reference in- 
formation. Then, for the application reference informa- 
tion, it inputs the same to the domain conversion unit 
652 to the node name conversion unit 655 again in ac- 
cordance with the contents thereof. By repeating such *o 
processing, the conversion control unit 651 controls the 
system so that the finally input application reference in- 
formation is indicated by the system reference informa- 
tion and outputs the obtained system reference informa- 
tion. •*$ 

Note that, this conversion control unit 651 has a 
storage means and sequentially stores the system ref- 
erence information for output sequentially detected with 
respect to the input reference information. Then, when 
all reference information are finally converted to the sys- so 
tern reference information, it reads the contents of this 
storage means and outputs the convened system refer- 
ence information together. 

Further, in the conversion control unit 65 1 . when ad- 
ditional information such as information designating the 55 
commmication media are input together with the appli- 
cation reference information, itoutputsalsotheaddition- 
al information to the conversion unit together with the 



reference information. 

The domain conversion unit 652 converts the. input 
domain reference information to the set of elements 
composing the domain by referring to the domain refer- 
ence table 53. More specilically, it converts this to for 
example a set of the entity reference and the domain 
reference contained in that domain. 

The entity conversion unit 653 converts the input 
entity reference information to the reference information 
of the process in which that entity exists and the remote 
identifier there by referring to the entity reference table. 
This entity reference table is substantially the same ta- 
ble as the local function reference table 50 and remote 
function reference table 51 mentioned before. Note that, 
here, "entity" indicates for example a function, instance, 
class, method, or global variable. 

The process conversion unit 654 converts the input 
process reference information to the information of the 
node at which that process exists and the process iden- 
tifier in that node by referring to the process reference 
table 52. The information of the node is either of the DNS 
name, IP address, or uniquely given node name. 

The node name conversion unit 655 converts the 
input node name to the DNS name or the IP address by 
referring to the node reference table contained in the 
process reference table 52. 

The DNS proxy conversion unit 656 converts the 
input DNS name to the IP address. The DNS name 
proxy conversion unit 556 has a communication means 
and storage means inside it. Then, when the DNS name 
is input, it communicates with the DNS name conversion 
unit 160. that is, the external reference-conversion de- 
vice, via the communication means and requests the 
conversion of the DNS name to the IP address. Then, it 
outputs the result of this as the conversion result. 

Further, in the DNS name proxy conversion unit 
656. the correspondence between the DNS name and 
the IP address obtained at this time is stored in the in- 
ternal storage means mentioned before. Then, when the 
DNS name to be converted is input, first, the DNS 
names stored in this storage means are searched. 
When there is the same DNS name (which has been 
already once converted), the IP address stored in that 
storage means is output as the conversion result. Ac- 
cordingly, only a DNS name not stored in the internal 
storage meam is inquired about to the DNS name con- 
version unit 160. 

Note that, in the converters of the domain conver- 
sion unit 652 to the node name conversion unit 655, 
when the input application reference information is a 
name or ID. the corresponding items of the tables are 
searched by this by appropriately referring to the domain 
name hash table 54 to the remote function ID hash table 
56 and not illustrated other hash table. 

Further, the conversion of the information regarding 
the node on the network performed in the process con- 
version unit 654 to the node name conversion unit 655 
is carried out by referring to the additional information 
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indicating the communication media input from the con- 
version control unit 651 together with the reference in- 
formation. This is because, if the communication media 
is different, usually the address is different even with re- 
■ spect to the same system. Accordingly, as the additional s 
information, concretely information designating the 
Ethernet or ATM is input. Note that, in the present em- 
bodiment, where the communication media is not des- 
ignated, the conversion is carried out by using the Ether- 
net as the communication media as a default. 

- Next, an explanation will be made of the concrete 
operation of this reference resolution unit 650 by refer- 
ring to Fig. 59. 

Figure 59 is a view of the reference information con- 
version processing when a message is transmitted to 
all entities belonging in the domain as one example of 
the operation of the reference converter 650. 

Note that, in Fig. 59, where a similar operation is 
carried out in the above reference conversion device 
530 of the first embodiment in each conversion process- 
ing, the type of the conversion unit performing the con- 
version processing and the conversion operation are si- 
multaneously described. 

First, when the domain name D1 is input to the con- 
version control unit 651 of the reference converter 650, 
since the content is the domain name, the conversion 
control unit 651 outputs the reference information there- 
of to the domain conversion unit 652. 

In the domain conversion unit 652, all entities and 
domains contained in a domain are extracted by refer- 
ring to the domain reference table 53 via the domain 
name hasb table 54^ As a result, a domain name D1 is 
converted to a set of entities E1 and E2 and domain 
names D2 and D3. 

For the domain names D2 and D3 among the ob- 
tained elements, the conversion is further carried out re- 
flexiveiy in the domain conversion unit 652. They are 
converted to a set of entities E3 and E4 and entities E5 
and E6 by this. 

As a result, the input domain name D is converted 
to the set of entities (E 1 , E2, E3, E4, E5 and E6) in the 
domain conversion unit 652. 

Next, the obtained entities E 1 to E6 are sequentially 
input to the entity conversion unit 653. In the entity con- 
version unit 653, the entity reference information E1 to 
E6 are converted to the set of the reference information 
and the remote identifiers of the process in which the 
entity exists. As a result, as illustrate, the reference in- 
formation indicated by the set of reference information 
P1 to P4 and remote identifiers ID1 to I D3 are obtained. 

Next, the conversion control unit 651 inputs the ob- 
tained reference information P1 to P4 of the process to 
the process conversion unit 654. In the process conver- 
sion unit 654, the reference information of that process 
is converted to the node information and process iden- 
tifier. Namely, the node information indicated by the 
DNS name and the process identifiers in that node are 
obtained with respect to the processes P1 and P2, the 



node information indicated by the IP address and the 
process identifier at that node are obtained with respect 
to the process P3, and the node information indicated 
by the node name and the process ID at that node are 
obtained with respect to the process P4. 

In the information of node name, the node informa- 
tion indicated by the node name obtained with respect 
to the process P4 is next input to the node name con- 
version unit 655 and converted to the DNS name. 

Further, the node information obtained as the DNS 
name obtained with respect to the processes P1 and P2 
is transferred to the DNS name conversion unit 560 via 
the node name conversion unit 655 and converted to 
the IP address at the DNS name conversion unit 560. 

As a result of such processing, the input domain 
name D1 is all converted to the system reference infor- 
mation indicated by the system address. 

In this way, the reference converter 650 performs 
the conversion of the reference information by recursive 
and hierarchical processing and therefore even on a 
complex computer network, the reference information 
of the application level can be converted to the system 
reference information. In other words, various compu- 
tation resources, that is the domains, nodes, processes, 
functions, instances, classes, methods, global varia : 
bles, etc. can be referred to from the logical application 
reference information without depending upon the sys- 
tem structure. As a result, a more sophisticated distrib- 
uted processing system not depending upon the physi- 
cal network structure can be realized. 

Note that, the present invention is not limited to the 
present embodiment and can be modified in any way. 

For example, the configuration of a reference con- 
verter for an embodiment of the invention was shown in 
Fig. 58, but this configuration shows the basic functions 
and may be modified in various ways. 

As one example thereof, an example of a more con- 
crete configuration of the reference converter is shown 
in Fig. 60. 

The reference resolver 700 shown in Fig. 60 con- 
stitutes the conversion unit by separating this to a name 
solution unit 702 and a reference solution unit 708. 

Below, an explanation will be made of the configu- 
ration and functions of the reference resolver 700. 

The reference resolver 700 has a solution unit man- 
agement unit 701 , a name solution unit 702, a reference 
solution unit 708, and a table nanagement unit 214. 

The solution unit management unit 701 performs 
the overall management of the reference solution unit 
700. Concretely, it decides whether the input reference 
information is the name or reference information. If it is 
the name, it outputs it to the name solution unit 702. and 
if it is the reference information, it outputs it to the refer- 
ence solution unit 708. 

The name solution unit 702 converts the name on 
the name space to be converted input from the solution 
unit management unit 701 to the corresponding refer- 
ence. 
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The name solution management unit 703 performs 
the overall management of the name solution unit 702. 
The name solution management unit 703 outputs the 
input name to one of the domain name solution unit 704 
to the proxy solution unit 707 in accordance with the type 
of the given name. Then, it reads the reference obtained 
at that solution unit and outputs the same to the solution 
unit management unit 701. 

The domain name solution unit 704 performs the 
conversion from the domain name to the domain refer- 
ence by utilizing the domain name hash table 54. 

The process name solution unit 705 performs the 
conversion from the process name to the process refer- 
ence by utilizing the function/process name hash table 
55. 

The entity name solution unit 706 performs the con- 
version from the entity name to the entity reference by 
utilizing a not illustrated other name hash table. The ex- 
ternal proxy solution unit 707 performs the conversion 
from the external name to the external reference by uti- 
lizing the proxy. Concretely, it is the proxy to for example 
the DNS name server. 

The reference solution unit 703 performs the con- 
version of the reference to be converted input from the 
solution unit management unit 701 to the primitive ref- 
erence. 

The reference solution management unit 708 per- 
forms the overall management of the reference solution 
unit 703. The reference solution management unit 709 
outputs the input reference to one of the domain refer- 
ence solution unit 710 to the external reference solution 
unit 713 in accordance with the type of the given refer- 
ence. Then, it reads the conversion result obtained at 
that solution unit and outputs this to the solution unit 
management unit 701. 

The domain reference solution unit 710 performs 
the conversion from the domain reference to the entity 
reference. Concretely the domain reference is convert- 
ed to the set of the (entity reference and domain refer- 
ence). Then, the domain reference obtained as the re- 
sult of conversion is converted reflexively in the domain 
reference solution unit 710. 

The process reference solution unit 711 performs 
the conversion from the process reference to the prim- 
itive process reference. At this time, by designating the 
communication media, the network node name is con- 
verted to the desired network address. 

The entity reference solution unit 712 performs the 
solution from the input entity reference to the primitive 
entity reference. In the present example, in the case of 
the conversion of the local entity reference, the given 
local entity reference is returned as it is as the conver- 
sion result. Further, in the case of the solution of the re- 
mote entity reference, the process reference appearing 
in the internal expression is converted to the primitive 
reference and the result is output. 

The external reference solution unit 713 performs 
the solution from the external reference to the primitive 



external reference. 

Note that, in Fig. 60. the external name solution unit 
715 is the DNS name converting means and converts 
the inputs DNS name to the IP address. 

5 Further, the message transmission unit S00 per- 
forms the message transmission for performing the 
communication among the objects in the distributed 
processing system 1 . 

Accordingly, in the reference resolver 700, con- 

io cretely the reference information indicating for example 
the node receiving the transmission, process, and ob- 
ject are input from the message transmission unit 800, 
and the reference information is converted to the system 
reference information at the reference solution unit 700 

is and output to the message tranmission unit 800. The 
message transmission unit 800 actually performs the 
communication based on the system reference informa- 
tion. 

The reference converter may also be configured in 
20 this way. 

Claims 



25 t. a parallel distributed processing system wherein a 
plurality of operation processing nodes for execut- 
ing processes provided with one or more computa- 
tion objects are connected with each other through 
a network, wherein 

30 

when calling and executing a computation ob- 
ject or its facility provided in a first process by 
a second process. 

said second process obtains location informa- 
35 tion directly specifying the computation object 

or its facility on the network from referring 
means using as a key a name or identifier of 
the computation object or its facility, transmits 
this location information to the first process. 
40 and calls up the computation object or its facility 

provided by the first process. 

2. A parallel distributed processing system as set forth 
in claim 1 , wherein 

said referring means is comprised of 
a local referring means provided in a proces re- 
ceiving a call for a computation object or its fa- 
cility for showing for the computation object or 
50 its facility which is called the correspondence 

between an identification number of the com- 
putation object or its facility and an execution 
address of the computation object or its facility 
and 

55 a remote referring means provided in a process 

originating a for a computation object or its fa- 
cility for showing for the computation object or 
its facility which it calls the correspondence be- 
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tween a name or identifier of the computation 
object or its facility and an identification number 
of the compulation object or its facility, 
a process originating a call uses the name or 
identifier of the computation object or its facility 5 
which it calls as a key to obtain a corresponding 
identification number from the remote referring 
means and sends a message containing the 
identification number to the process receiving 
the caJMor the computation object or its facility. io 
and 

the process receiving the call uses the identifi- 
cation number contained in the message sent 
from the process originating the call as a key to 
obtain the execution address of the computa* r5 
tion object or its facility which is called from the 
local referring means and executes the compu- 
tation object or its facility which is called based 
an' the execution address. 

20 

3. A parallel distributed processing system as set forth 
in claim 1. wherein 

a computation space is prescribed which is 
composed of a set of any of the one or a plurality 25 
of computation objects or their facilities and in 
which uniqueness of names or identifiers of the 
computation objects or their facilities is re- 
quired in only the set; and 

the referring means gives location information 30 
on a computation object or its facility by using 
the name owdentifier of the computation object 
or its facility in the computation space as a key. 

4. A parallel distributed processing system as set forth 35 
in claim 1 . wherein each operation processing node 
has: 

a process allocating means for allocating a pro- 
gram module at a predetermined operation *o 
processing node and generating a process at 
an operation processing node receiving the al- 
location based on allocation information indi- 
cating correspondence between location infor- 
mation of the program module for realizing the •** 
process and information of the operation 
processing node receiving the allocation or the 
program module and 

a reference information*generating means for 
generating the reference information of the so 
computation objects or their facilities which the 
program modules for realizing a process refers 
to each other based on the allocation informa- 
tion and the reference relationship among the 
computation objects or their facilities described « 
in the program module in each process. 

5. A parallel distributed processing system as set forth 



in claim 1. wherein a computation space is pre- 
scribed which is composed of a set of any of the one 
or a plurality of computation objects or their facilities 
and in which uniqueness of names or identifiers of 
the computation objects or their facilities is required 
in only the set; 

comprising a plurality of reference converting 
means for converting logical reference informa- 
tion specifying a computation object or its facil- 
ity in a computation space to either of similar 
logical reference information and system refer- 
ence information corresponding to the location 
information or reference information of a com- 
bination of the same, and 
a conversion control means for recursively in- 
putting the input logical reference information 
and the logical reference information converted 
and generated in the reference converting 
means to any one of the plurality of reference 
converting means based on the logical refer- 
ence information and converting the input log- 
ical reference information to the system refer- 
ence information: and 

converting the logical reference information of 
the process receiving the communication to 
system reference information by said reference 
converting means through said conversion 
control means and performing communication 
among processes. 

6. A parallel distributed processing system as set forth 
in claim 2. wherein the message includes a time in- 
dicated by a timer used in the process originating a 
call. 

7. A parallel distributed processing system as set forth 
in claim 2. wherein when a plurality of messages 
are sent from processes originating calls to a proc- 
ess receiving the call, the plurality of facility calls 
corresponding to these messages are sequentially 
made according to a priority order for processing at 
the process receiving the call included in the mes- 
sages. 

8. A parallel distributed processing system as set forth 
in claim 2. wherein the message includes a time 
when the facility which is called is to be executed in 
the process receiving the call. 

9. A parallel distributed processing system as set forth 
in any one of claims 1 to 8. wherein said computa- 
tion object or facility is any one of a file, function, 
class, instance, method, and global variable. 

1 0. A parallel distributed processing system as set forth 
in claim 5, wherein 
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said plurality of reference converting means is 
comprised of a first reference converting 
means for convecting the logical reference in- 
formation to the system reference information, 
a second reference converting means for con- s 
verting the logical reference information to a set 
of the system reference information, a third ref- 
erence converting means for converting the 
logical reference information to other logical 
reference information, a fourth reference con* 10 
verting means for converting the logical refer- 
ence information to a set of the other logical ref - 
erence information, and a fifth converting 
means for converting the logical reference in- 
formation to a set of the system reference in- is 
formation and the logical reference information; 
and 

the conversion control means inputs the input 
logical reference information and the logical 20 
reference information generated by conversion 
to any one of the first reference converting 
means to the fifth reference convening means 
based on the contents of the logical reference 
information. 25 

11. A parallel distributed processing system as set forth 
in any one of claims 1 to 10, wherein when a new 
computation object or facility is registered, the com- 
putation object or facility which will be called from 20 
the process and which the process will call are an- 
alyzed and the reference information are dynami- 
cally prepared based on the result of the analysis. 

12. An operation processor having for executing a proc- j5 
ess provided with one or a plurality of computation 
objects and connected with other operation proces- 
sors through a network, comprising: 

a process generating means for generating a -*o 
first process corresponding to a first program 
module, 

a reference information generating means for 
generating reference information relating to a 4 $ 
location information directly specifying a com- 
putation object or its facility on a network re- 
ferred to by a program module for realizing a 
process based on a reference relationship 
among the computation objects or their facili- so 
ties described in the program module, 

a reference information holding means for hold- 
ing said reference information, 

55 

a reference inquiry means for making an inquiry 
to said reference information holding means 
using a name or identifier of said computation 



object or its facility as a key thereby obtaining 
the execution address of the computation ob- 
ject or its facility on the network from said ref- 
erence information when said first process calls 
and executes a computation object or its facility 
provided in a second process corresponding to 
a second program module, and 

a message communicating means for tranmit- 
ting and receiving a message including the lo- 
cation information obtained by said reference 
inquiry means between said first process and 
said second process and performing process- 
ing for calling the computation object or its fa- 
cility provided by said first process and said 
second process. 

1 3. An operation processor having for executing a proc- 
ess provided with one or a plurality of computation 
objects and connected with other operation proces- 
sors through a network, comprising: 

a process allocating means for substantially 
generating the process by indicating the alloca- 
tion of a program module and a generation of 
a process corresponding to said program mod- 
ule with respect to a predetermined operation 
processor based on an allocation information 
indicating correspondence between location in- 
formation of a program module for realizing the 
process and information of the operation proc- 
essor receiving the allocation of the program 
module, 

a reference information generating means for 
substantially generating reference information 
by indicating the generation relating to the lo- 
cation information directly specifying a compu- 
tation object or its facility to which a plurality of' 
program modules refer each other with respect 
to said process which is generated correspond- 
ing to the program module based on a refer- 
ence relationship among the computation ob- 
jects or their facilities described in the program 
module, and 

a message communicating means for indicat- 
ing an allocation of the process with respect to 
said operating processor and indicating the 
generation of said reference information with 
respect to the process. 

14. A parallel distributed processing method in a net- 
work wherein a plurality of operation processing 
nodes for executing processes provided with one or 
more computation objects are connected with each 
other wherein 

when calling and executing a computation ob- 
ject or its facility provided in a first process by 
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a second process, 

said second process obtains location informa- 
tion directly specifying the computation object 
or its facility on the network from reference in- 5 
formation using as a key a name or identifier of 
the computation object or its facility, transmits 
this location information to the first process, 
and calls up the computation object or its facility 
provided by the first process. io 

1 5. A parallel distributed processing method as set forth 
in claim 14, wherein 

said reference information comprises '5 
local reference information provided in a proc- 
ess receiving a call for a computation object or 
its facility for showing for the computation ob- 
ject or its facility which is called the correspond- 
ence between an identification mumber of the 20 
computation object or its facility and an execu- 
tion address of to computation object or its fa- 
cility and 

remote reference information provided in a 
process originating a call for a computation ob- 25 
ject or its facility for showing for the computa- 
tion object or its facility which it calls the corre- 
spondence between a name or identifier of the 
computation object or its facility and an identi- 
fication number of the computation object or its so 
facility. 

a process originating a call uses the name or 
identifier of the computation object or its facility 
which it calls as a key to obtain a corresponding 
identification number from the remote referring 35 
means and sends a message containing the 
identification number to the process receiving 
the call for the computation object or its facility, 
and 

the process receiving the call uses the identifi- *o 
cation number contained in the message sent 
from the process originating the call as a key to 
obtain the execution address of the computa- 
tion object or its facility which is called from the 
local referring means and executes the compu- J 5 
tation object or its facility which is called based 
on the execution address. 

1 6. A parallel distributed processing method as set forth 

in claim 14, wherein 50 

a computation space is prescribed which is 
composed of a set of any of the one or a plurality 
of computation objects or their facilities and in 
which uniqueness of names or identifiers of the 55 
computation objects or their facilities is re- 
quired in only the set: and 
the reference information gives location infor- 



mation on a computation object or its facility by 
using the name or identifier of the computation 
object or its facility in the computation space as 
a key. 

17. A parallel distributed processing system as set forth 
in claim 14, wherein each operation processing 
node 

allocates a program module at a predetermined 
operation processing node and generates a 
process at an operation processing node re- 
ceiving the allocation based on allocation infor- 
mation indicating correspondence between lo- 
cation information of the program module for re- 
alizing the process and information of the op- 
eration processing node receiving the alloca- 
tion of the program module and 

generates the reference information of the 
computation objects or facilities which the pro- 
gram modules for realizing a process refers 
each other to based on the allocation informa- 
tion and the reference relationship among the 
computation objects or facilities described in 
the program module in each process. 

1 8. A parallel distributed processing method as set forth 
in claim 14, comprising 

defining a computation space which is com- 
posed of a set of any of the one or a plurality of 
computation objects or their facilities and in 
which uniqueness of names or identifiers of the 
computation objects or their facilities is re- 
quired in only the set: 

determining the reference conversion method 
for the logical reference information specifying 
a computation object or its facility receiving the 
communication of the process in the computa- 
tion space based on its content: 

converting the logical reference information re- 
ceiving the communication to either of other 
logical reference information and system refer- 
ence information directly specifying an object 
on the computation network or a combination 
of the same by the determined reference con- 
version method; 

determining the method of the reference con- 
version again for the logical reference informa- 
tion among the reference information resulting 
from the conversion based on the logical refer- 
ence information: 

performing the conversion again by the deter- 
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mined method reference information: 

repeatedly perfoming the determination of the 
reference conversion method with respect to 
the logical reference information and the con- 5 
version based on this until all elements of the 
reference information resulting from the con- 
version become the system reference informa- 
tion and, when all elements become the system 
reference information, communicating among io 
processes based on the reference information ; 

whereby the processes perform distributed 
processing substantially linked up. 

is 

1 9. A parallel distributed processing method as set forth 
in claim 15, wherein the message includes a time 
indicated by a timer used in the process originating 
a call. 

20 

20. A parallel distributed processing method as set forth 
in claim 15. wherein when a plurality of message 
are sent from processes originating calls to a proc- 
ess receiving the call, the plurality of facility calls 
corresponding to these messages are sequentially 25 
made according to a priority order for processing at 

the process receiving the call included in the mes- 
sages. 

21. A parallel distributed processing method as set forth 30 
in claim 15, wherein the message includes a time 
when the facility which is called is to be executed in 

the process receiving thexall. 

22. A parallel distributed processing method as set forth 35 
in any one of claims 14 to 23. wherein the compu- 
tation object or its facility is any one of a function, 
class, instance, method, global variable, and file. 

23. A parallel distributed processing method as set forth -o 
in claim 18, wherein the method of the reference 
conversion is determined from among a first meth- 
od for converting logical reference information to 

the system reference information, a second method 
for converting the logical reference information to a •*$ 
set of the system reference information, a third 
method for converting the logical reference informa- 
tion to other logical reference information, a fourth 
method for converting the logical reference informa- 
tion to a set of the other logical reference informa- so 
tion, and a fifth method for converting the logical ref- 
erence information to a set of the system reference 
information and the logical reference information 
based on the contents of the input logical reference 
information and the logical reference information 55 
generated by conversion. 

24. A parallel distributed processing method as set forth 



in any one of claims 14 to 23, wherein when a new 
computation object or facility is registered, the com- 
putation object or facility which will be called from 
the process and which the process will call are an- 
alyzed and the reference information are dynami- 
cally prepared based on the result of the analysis. 
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